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Executive  Summary 

3400  level  Career  and  Technical  Studies  (CTS)  courses  were  developed  to  be  taken  by  Alberta 
high  school  students  who  have  the  prerequisite  skills  and  interest  required  to  complete  first 
period  apprenticeship  courses  (3400  series).  These  1st  period  apprenticeship  courses  permit  a 
student  to  work  towards  journeyman  credentialing  in  a  specific  trade  area,  which  is  explained  in 
detail  in  the  Alberta  Journeyman  Certification  Explanation  section  of  this  study  on  P.  13. 

Red  Deer  College  (RDC)  under  the  Regional  Trades  Initiative  has  developed  a  program  for  CTS 
teachers  and  students  that  provides  (1)  professional  learning  opportunities  for  certified  teachers 
to  improve  upon  their  practical  skills  and  theoretical  knowledge  in  carpentry  and  welding,  and 
(2)  a  separate  and  standardized  student  assessment  tool  of  the  theoretical  component  of  the  3400 
1st  period  apprenticeship  program  in  carpentry  and  welding  using  The  Learning  Manager  (TLM) 
online  platform.  This  study  focuses  upon  use  of  TLM  as  a  standardized  assessment  tool.  A 
detailed  explanation  of  the  background  of  TLM  is  provided  in  Appendix  A  on  P.  33  of  this  study. 

In  February  2009  the  System  Improvement  Group,  Alberta  Education  and  RDC,  began  a  pilot 
using  a  small  number  of  high  schools  that  used  the  TLM  Online  Student  Assessment  Program. 
TLM  had  been  adapted  by  RDC  for  carpentry  and  welding  in  the  new  3400  1st  period 
apprenticeship  courses.  This  pilot  was  conducted  to  determine  the  relevance,  benefits  and 
challenges  of  high  school  student  online  standardized  assessment  using  a  carpentry  and  welding 
apprenticeship  model  that  follows  the  second  component  of  the  RDC  program  described  above. 

The  pilot  was  based  on  the  current  CTS  Program  of  Studies  (POS)  and  mapped  the  new  3400  1st 
period  courses  that  are  to  be  implemented  in  September  2009,  as  TLM  is  built  using  the  3400 
courses.  Students  completed  3400  level  courses  and  were  credited  with  courses  from  the  current 
POS.  The  pilot  was  intended  to  examine  the  standardized  assessment  opportunities  of  TLM  and 
not  the  new  3400  level  courses.  A  map  of  the  current  POS  courses  and  the  3400  level  courses  is 
provided  in  Appendix  B  on  P.  38  of  this  study. 

System  Improvement  Group  through  work  on  the  CTS  Student  Assessment  Framework  Project  as 
part  of  the  larger  CTS  Initiative  managed  by  Curriculum  Branch,  Program  and  Development 
Standards  Division,  Alberta  Education  has  collaborated  with  RDC  to  pilot  TLM  in  four  high 
schools  and  one  outreach  school.  This  pilot  was  offered  to  determine  the  relevance,  benefits  and 
challenges  of  high  school  student  online  standardized  assessment  using  a  carpentry  and  welding 
apprenticeship  model. 

The  specific  areas  that  this  online  student  assessment  pilot  was  to  examine  included: 

1.  Determining  the  impact  of  standardized  student  assessment  upon  opportunities  for  student 
success  which  potentially  narrows  the  gap  between  schools  of  subjective  teacher-awarded 
marks. 

2.  Determining  the  value  of  this  online  assessment  tool  for  potential  use  by  non-journeyman  CTS 
teachers. 

3.  Assessing  the  impact  of  this  online  assessment  tool  upon  student  achievement. 

4.  Assessing  the  impact  upon  student  and  teacher  perception  of  the  online  tool. 

5.  Determining  the  potential  for  future  expansion  of  this  tool  beyond  the  initial  pilot  schools. 
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6.   Determining  the  value  of  a  specific  student  achievement  measure  in  the  Student  Assessment 
Framework. 


The  study  Key  Findings  and  Recommendations  are  presented  in  a  chart  format  to  indicate  their 
relative  correspondence. 


Study  Key  Findings 

Recommendations 

The  pilot  conducted  for  this  study  had  a  small 
number  of  schools;  teachers  and  students 
participating  that  limited  the  available  data  and 
consequently  the  overall  findings. 

That  Alberta  Education  pursues  a  partnership 
with  Red  Deer  College  to  expand  use  of  The 
Learning  Manager  online  student  assessment 
program  into  a  larger  pilot  opportunity  with 
additional  high  schools. 

Some  participating  pilot  study  teachers 
expressed  an  interest  in  continued  use  of  the 
TLM  online  standardized  student  assessment 
tool  regardless  of  the  status  of  financial  support 
for  the  resource  from  Alberta  Education. 

That  Red  Deer  College  examine  opportunities 
to  provide  The  Learning  Manager  to  school 
authorities  in  Alberta  through  agreements  as  a 
tool  to  provide  additional  resources  and 
standardized  assessment  for  CTS  teachers  at 
the  3400  level  of  programming. 

Teachers,  both  pilot  study  teachers  and 
teachers  in  subject  areas  at  the  study  schools 
not  associated  with  the  study,  expressed 
interest  in  expanding  the  TLM  standardized 
student  assessment  tool  to  trade  areas  beyond 
carpentry  and  welding  and  academic  subjects. 

That  Red  Deer  College  and/or  other  post- 
secondary  institutions  continues  work  to 
expand  TLM  beyond  the  two  trades  areas  of 
carpentry  and  welding  examined  in  this  pilot 
into  other  1  st  period  apprenticeship  programs. 

Teachers  expressed  interest  in  expanding  the 
TLM  standardized  assessment  tool  to  CTS 
courses  at  the  introductory,  intermediate  and 
advanced  levels  to  assist  in  addressing  the 
significant  grading  differences  and  the  gap 
between  students'  earned  marks  and  their 
actual  abilities. 

That  Alberta  Education  and  Red  Deer  College 
examine  the  opportunity  to  expand  The 
Learning  Manager  online  student  assessment 
program  or  other  online  student  assessment 
program  to  CTS  courses  at  the  introductory, 
intermediate  and  advanced  levels. 

There  are  challenges  experienced  by  schools  in 
providing  facilities  capable  of  providing 
courses  at  the  3400  level  1st  period  apprentice 
and  hiring  teachers  who  are  also  certified 
journeyman  and  able  to  teach  to  that  level. 

That  Alberta  school  authorities  be  actively 
encouraged  by  Alberta  Education  to  explore 
partnerships  and  shared  facility  agreements 
with  neighboring  school  authorities  and  post- 
secondary  institutions  to  more  effectively 
utilize  programs,  facilities,  resources  and 
personnel. 
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A  majority  of  CTS  teachers  are  not  certified  at 
the  journeyman  level  and  therefore  those 
students  that  register  in  3400  level  courses  with 
these  teachers  are  unable  to  challenge  1st 
period  journeyman  exams  through 
Apprenticeship  and  Industry  Training  (AIT), 
Alberta  Advanced  Education  and  Technology. 

That  Alberta  Education  continue  dialogue  with 
Apprenticeship,  Industry  and  Training  (AIT), 
Alberta  Advanced  Education  and  Technology 
to  address  the  status  of  students  who  complete 
ine  jtuu  icvei  courses  inrougn  inc  insiruciion 
offered  by  a  non-journeyman  CTS  teacher  and 
their  ability  to  write  1st  period  journeyman 
exams. 

The  mandate  of  Red  Deer  College  to  serve  the 
Central  Alberta  region  and  the  number  of 

aVdlldUlc        pei&umiei        allU  ICCIlIlUlUglCal 

resources  that  they  currently  possess  pose  a 
challenge  for  the  possible  expansion  of  TLM  to 
additional  high  schools 

That  Alberta  Education  explores  the 
opportunity  with  Red  Deer  College  to  assume  a 

ff^intvci  1  1  f ~» q riprt li i 1"\  t~/~\  1  o  i ti  trip  TiitiiT"*^  ncn  r\T  Tn*^ 
LLlHIal  ICaUCI Mlip  IU1C  111  II1C  lUlUIC  USL  Ul   1  lie 

Learning  Manager  online  student  assessment 
program  through  direct  management  of  the 
process. 
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CTS  Student  Online  Assessment  Pilot  Study:  An  Exploration  of  The  Learning  Manager 
(TLM)  Model  With  Red  Deer  College 

Background 

Red  Deer  College  under  the  Regional  Trades  Initiative  has  developed  a  program  for  CTS 
teachers  and  students  that  provides  (1)  professional  learning  opportunities  for  certified  teachers 
to  improve  upon  their  practical  skills  and  theoretical  knowledge  in  carpentry  and  welding,  and 
(2)  a  separate  and  standardized  student  assessment  tool  of  the  theoretical  component  of  the  3400 
1st  period  apprenticeship  program  in  carpentry  and  welding  using  The  Learning  Manager  (TLM) 
online  platform.  This  study  focuses  upon  use  of  TLM  as  a  standardized  assessment  tool. 

This  program  was  originally  field-tested  by  RDC  in  2007  in  carpentry  and  welding  at  a  small 
number  of  rural  high  schools  in  the  Red  Deer  area  to  sample  five  Individual  Learning  Modules1 
(ILMs)  which  make-up  a  component  of  the  overall  trade  area.  Teachers  observed  at  that  time 
that  the  program  was  good,  but  the  developed  ILMs  were  too  limiting  for  their  students.  This 
pilot  expanded  the  number  of  ILMs,  schools,  teachers  and  students. 

System  Improvement  Group,  Accountability  and  Reporting  Division,  Alberta  Education  through 
work  on  the  CTS  Student  Assessment  Framework  Project  as  part  of  the  larger  CTS  Initiative  has 
collaborated  with  RDC  to  pilot  TLM  in  four  high  schools  and  one  outreach  school.  This  pilot 
was  offered  to  determine  the  relevance,  benefits  and  challenges  of  high  school  student  online 
standardized  assessment  using  a  carpentry  and  welding  apprenticeship  model. 

This  pilot  began  partly  in  response  to  observations  made  by  trade  instructors  at  the  Southern 
Alberta  Institute  of  Technology  (SAIT)  in  focus  groups  held  in  September  2008  who  stated: 

"High  school  students  coming  from  some  schools  in  the  province  who  have  apparently 
earned  high  marks  in  CTS  modules  do  not  have  higher  level  skills  or  knowledge 
comparable  to  students  from  other  high  schools  who  apparently  earned  lower  marks.  It 
is  evident  that  the  grading  of  high  school  CTS  modules  by  a  number  of  teachers  leaves  a 
lot  to  be  desired  and  they  are  significantly  inflating  their  students '  marks  which  hurts 
those  students  when  they  get  to  our  programs  and  it  becomes  evident  that  they  are  not 
competent  to  the  degree  that  their  high  school  CTS  grades  indicate  that  they  should  be.  " 

The  above  observation  provided  by  SAIT  instructors  is  supported  by  high  school  CTS  instructors 
gathered  through  individual  discussions  as  part  of  this  pilot  project.  These  observations  will  be 
further  discussed  later  in  this  document. 

As  indicated,  this  pilot  is  a  component  of  the  larger  CTS  Initiative  that  is  being  managed  by 
Curriculum  Branch  in  Program  Development  and  Standards  Division,  Alberta  Education. 


Individual  Learning  Modules  (ILMs)  are  hardcopy  booklet  resources  produced  by  Apprenticeship  and  Training 
(AIT),  Alberta  Advanced  Education  and  Technology  for  each  individual  apprenticeship  training  module.  This  is 
further  explained  in  the  Individual  Learning  Module  (ILMs)  Hardcopy  Resources  section  of  this  study  on  P.  12. 
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CTS  Student  Online  Assessment  Pilot  Project  Study  Description 

Through  collaboration  between  the  Regional  Trades  Initiative  at  RDC,  Program  Development 
and  Standards  Division  and  Accountability  and  Reporting  Division  entered  into  a  charter 
agreement  to  examine  TLM  and  specific  measures  of  the  CTS  Student  Assessment  Framework. 
The  specific  areas  that  this  online  student  assessment  pilot  was  to  examine  included: 

1.  Determining  the  impact  of  standardized  student  assessment  upon  opportunities  for  student 
success  which  potentially  narrows  the  gap  between  schools  of  subjective  teacher-awarded 
marks. 

2.  Determining  the  value  of  this  online  assessment  tool  for  potential  use  by  non-journeyman 
CTS  teachers. 

3.  Assessing  the  impact  of  this  online  assessment  tool  upon  student  achievement. 

4.  Assessing  the  impact  upon  student  and  teacher  perception  of  the  online  tool. 

5.  Determining  the  potential  for  future  expansion  of  this  tool  beyond  the  initial  pilot  schools. 

6.  Determining  the  value  of  a  specific  student  achievement  measure  in  the  Student 
Assessment  Framework. 

In  order  to  better  understand  the  opportunities  and  challenges  of  a  program  such  as  TLM  and  the 
possible  impact  upon  student  achievement,  the  following  data  sources  were  accessed  and 
activities  undertaken  in  the  preparation  of  this  study  document: 

•  Discussions  with  representatives  from  the  Regional  Trades  Initiative,  RDC 

•  Focus  group  with  instructors  at  SAIT 

•  Participation  in  TLM  professional  learning  sessions  with  CTS  teachers 

•  Interviews  with  pilot  school  administrators  and  CTS  teachers 

•  Informal  discussions  with  CTS  students 

•  Survey  results  from  eight  school  authorities'  central  office  representatives 

•  Survey  results  from  pilot  school  CTS  teachers  and  students 

•  Student  course  marks 

•  Pilot  school  course  completions  in  carpentry  and  welding 

•  Examination  of  existing  TLM  programs  in  secondary  and  post-secondary  environments 

A  small  number  of  primarily  rural  high  schools  and  their  CTS  teachers  participated  in  this  pilot 
study.  The  schools  and  school  authorities  that  participated  included: 

•  Strathmore  High  School,  Golden  Hills  Regional  Division 

•  Trochu  Valley  School,  Golden  Hills  Regional  Division 

o    Trochu  Valley  Outreach  -  which  serves  three  area  high  schools 

•  Ecole  Secondaire  Lacombe  Composite  High  School,  Wolf  Creek  Regional  Division 

•  Ponoka  Composite  High  School,  Wolf  Creek  Regional  Division 

CTS  teachers  from  four  schools  were  approached  by  representatives  from  the  Regional  Trades 
Initiative  regarding  their  possible  participation  in  this  pilot  study  based  upon  their  physical 
location  within  central  Alberta  or  their  previous  collaboration  with  RDC.  The  outreach  school 
joined  in  the  pilot  through  their  work  with  students  at  Trochu  Valley  School. 
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The  participating  CTS  teachers  included  four  educators  who  are  certificated  as  journeyman  in 
either  welding  or  carpentry  and  an  additional  three  teachers  who  were  not  certificated  as 
journeyman.  The  non-journeyman  teachers  included: 

•  One  CTS  teacher  in  a  specialized  welding  lab  in  an  small  urban  setting 

•  One  CTS  teacher  in  a  general  course  delivery  lab  in  a  small  rural  setting 

•  One  Outreach  teacher/administrator  with  limited  CTS  experience  in  a  small  rural  setting 

Study  Limitations 

The  major  limitation  to  the  collected  data  is  the  small  number  of  participants  at  the  five  schools. 
This  pilot  was  small  in  scope  and  as  a  result,  there  were  only  five  participating  schools  with 
seven  participating  CTS  teachers  and  approximately  thirty-seven  (37)  students  involved.  A 
small  number  of  additional  schools  were  invited  to  participate  though  they  were  unable  to  accept 
the  offer  due  to:  not  having  students  who  were  eligible  to  register  for  3400  level  courses,  not 
having  CTS  courses  in  carpentry  or  welding  scheduled  for  the  pilot  time  frame  in  semester  2,  or 
the  CTS  teachers  and  school  administrators  felt  that  they  did  not  have  the  time  to  prepare  for  a 
pilot  initiative. 

This  creates  a  situation  in  which  a  few  people  are  unavoidably  able  to  increase  the  value  of  their 
responses.  The  number  of  participating  schools  and  consequently  of  participating  students,  was 
low  for  the  following  reasons: 

•  The  timeframe  with  which  this  pilot  developed  from  proposal  to  implementation  was 
short 

•  Departmental  budget  considerations 

•  Limited  personnel  and  information  technology  (IT)  resources  at  RDC  within  the  budget 
constraints  of  this  pilot 

•  RDC's  operational  mandate  to  stay  within  the  boundaries  of  Central  Alberta 

•  Pilot  intentional  design  to  include  small  rural  schools 

•  The  small  number  of  high  school  students  who  were  eligible  to  register  for  the  3400  1st 
period  apprenticeship  courses  when  this  pilot  developed 

The  online  survey  provided  to  participating  teachers  and  students  received  a  small  number  of 
responses:  five  teachers  and  three  students.  Five  of  seven  possible  teacher  responses  are  strong 
in  comparison  to  the  overall  number  of  participating  teachers,  though  the  overall  number  is  low. 
This  was  a  known  factor  upon  the  initiation  of  this  study.  As  a  result  of  this  small  n,  the  survey 
results  are  not  provided  in  this  document  nor  were  they  used  to  directly  impact  the  findings  of 
this  research.  Survey  results  were  used  to  provide  discussion  topics  for  the  formal  and  informal 
contacts  made  with  teachers,  school  administrators  and  students. 

Some  students  were  enrolled  in  the  pilot  by  their  participating  teacher  after  the  start  of  semester  2 
and  did  not  necessarily  have  the  same  amount  of  instructional  time  as  other  students.  This  is 
evident  for  carpentry  which  was  included  in  the  pilot  at  'the  last  moment'  in  order  to  expand  the 
TLM  opportunities  and  assist  in  determining  the  capabilities  of  TLM  in  a  different  trade  area. 
As  a  result  of  this  expansion,  carpentry  students  and  their  teachers  were  exposed  to  TLM  though 
they  were  not  able  to  explore  the  program  as  deeply  as  those  students  in  welding. 
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The  participation  in  this  study  of  three  non-journeyman  CTS  teachers  is  not  viewed  as  a 
limitation.  The  hypothesis  proposed  holds  that  all  CTS  teachers  may  benefit  from  use  of  the 
TLM  program  since  TLM  as  a  tool  provides  resources  and  standardized  student  assessment, 
resources  that  CTS  teachers  may  not  normally  have  access  to,  nor  expertise  in.  A  significant 
number  of  CTS  teachers  in  Alberta  do  not  possess  journeyman  credentials  and  this  is  identified 
in  the  Non-Journeyman  CTS  Teachers  section  of  this  study. 

The  3400  level  courses  currently  resourced  by  TLM  are  to  be  provided  through  the  instruction  of 
journeyman  certificated  instructors/teachers,  whereas  the  introductory,  intermediate  and 
advanced  CTS  courses  (non-3400)  do  not  require  journeyman  instructors.  The  potential  exists  to 
expand  TLM  to  other  CTS  courses. 

Pilot  students  using  TLM  are  in  two  categories,  those  completing  welding  and  those  in  carpentry. 
The  number  of  carpentry  courses  completed  at  the  3400  level  was  very  low  in  relation  to  the 
number  of  students  who  were  both  interested  in  the  program  and  for  whom  pilot  teachers  felt 
were  capable  of  attempting  3400  level  courses  successfully. 

There  were  several  reasons  for  the  low  number  of  credits  earned  in  carpentry  as  displayed  in 
chart  form  on  the  following  page: 

1.  Originally  only  welding  was  to  be  part  of  the  pilot.  Carpentry  was  incorporated  'at  the  last 
minute '  due  to  interest  expressed  by  some  of  the  pilot  schools  that  offered  both  carpentry  and 
welding  and  the  opportunity  that  the  pilot  presented  for  a  'test  vehicle'  for  carpentry. 

2.  The  RDC  carpentry  department  rushed  to  align  first  year  apprenticeship  to  3400  level  courses 
on  the  TLM  system  and  this  resulted  in  some  delays  for  start-up. 

3.  Many  of  the  carpentry  students  were  taking  full  course  loads,  so  this  pilot  became 
overwhelming  for  some  of  the  students. 

4.  The  delay  in  introducing  the  pilot  after  semester  2  had  already  started  may  have 
overwhelmed  students  who  already  had  a  full  course  load.  Their  initial  enthusiasm  for  TLM 
may  have  been  tempered  as  they  became  increasingly  aware  of  the  program  work 
requirements. 

5.  One  of  the  two  carpentry  teachers  in  this  pilot  was  actively  involved  in  Skills  Canada 
competitions  at  the  provincial  and  national  levels  and  this  was  the  focus  of  his  efforts.  As  a 
result,  the  pilot  may  have  not  received  the  emphasis  required  at  that  particular  site. 

6.  One  carpentry  teacher  was  not  familiar  with  digital  technology  and  was  not  comfortable  with 
online  applications.  RDC  personnel  spent  additional  time  working  with  this  individual. 

7.  This  pilot  was  an  'add-on'  to  the  regular  CTS  programming.  The  two  pilot  schools  that 
offered  carpentry  did  not  have  an  opportunity  to  adequately  timetable  the  instructional  hours 
necessary  for  students  in  addition  to  their  regular  timetable  programming  for  other  CTS 
courses. 

The  challenges  experienced  in  carpentry  were  not  evident  in  welding.  Welding  students  were 
able  to  progress  at  a  faster  pace  that  permitted  more  students  to  earn  credits  and  to  more 
thoroughly  test  the  TLM  assessment  program. 

Charts  1  and  2  indicate  the  number  of  credits  earned  by  each  student  participating  in  the  pilot.  It 
is  interesting  to  note  that  the  fourteen  welding  students  were  divided  amongst  five  pilot  teachers 
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and  the  twenty-three  carpentry  students  were  registered  by  only  two  pilot  teachers.  This  may  be 
an  indication  that  the  carpentry  teachers  and  their  students  were  overly  enthusiastic  about  the 
TLM  opportunity  and  did  not  fully  anticipate  the  challenges  of  participating  in  this  opportunity. 


Chart  1:  TLM  Earned  Carpentry  Credits 
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Chart  2:  TLM  Earned  Welding  Credits 
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The  lack  of  credits  earned  in  carpentry  by  the  number  of  students  participating  may  be  a  further 
indication  of  the  challenges  evident  with  the  required  learning  to  the  1st  period  apprenticeship 
level  in  the  3400  courses. 
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Non-Journeyman  CTS  Teachers 

It  is  generally  accepted,  though  the  exact  numbers  for  the  province  are  not  known  by  Alberta 
Education,  that  the  majority  of  CTS  teachers  in  Alberta  do  not  have  journeyman  status  in  any 
trade  area.  This  belief  is  supported  by  data  from  a  survey  of  superintendents  conducted  by  the 
System  Improvement  Group  in  collaboration  with  Workforce  Planning  Branch,  People 
Resources  Division,  in  November  2008  in  eight  school  authorities  in  Alberta  that  ranged  from  a 
small  rural  board  of  approximately  1500  students  to  a  large  urban  board  of  30,000  students. 
Results  of  this  survey  completed  by  superintendents  or  their  delegated  representatives  indicate 
that  of  the  total  226  teachers  who  were  identified  as  teaching  CTS  in  those  school  authorities, 
only  51  or  22.6%  of  them  were  known  by  central  office  personnel  to  be  certified  journeyman  in  a 
trade  area.  An  additional  45  teachers  or  19.9%  had  additional  non-journeyman  credentials  in 
other  trade  areas;  of  this  number  some  credentials  were  in  curricular  areas  in  which  earning 
journeyman  status  is  not  possible  (i.e.  business  administration,  agriculture,  transportation  and 
health  care). 

The  survey  of  superintendents  indicated  that  all  eight  school  authorities  have  experienced 
significant  challenges  in  recruiting  CTS  teachers  certificated  to  the  journeyman  level  and  they 
anticipate  that  this  challenge  will  increase  even  further  in  the  coming  years. 

Non-journeyman  teachers  are  not  currently  able  to  instruct  students  at  any  level  of  journeyman 
without  the  full-time  supervision  of  a  journeyman  instructor,  if  those  students  wish  to  then 
challenge  the  1st  period  apprenticeship  exams.  The  situation  requires  schools  and  school 
authorities  to  develop  partnerships  with  post-secondary  institutions  that  may  be  able  to  supply 
journeyman  instructors  who  will  assist  non-journeyman  CTS  teachers  or  perhaps  other 
partnerships  with  companies  or  individuals  in  local  communities  who  have  journeyman  status 
and  are  willing  to  assist  the  schools  under  the  supervision  of  certificated  teachers  who  are  not 
journeyman.  This  situation  was  explored  in  this  pilot  as  RDC  provided  a  journeyman  welding 
instructor  to  assist  a  non-journeyman  CTS  teacher  at  one  high  school  on  a  part-time  basis. 

TLM  as  Offered  by  Red  Deer  College 

The  TLM  model  developed  by  RDC  offers  some  specific  features  for  the  five  schools  involved 
in  this  pilot: 

•    Online  review  testing  for  each  unit  (screen  shot  provided  in  Appendix  C) 

o  All  online  testing  is  assessed  immediately  and  the  results  are  provided  to  the  student 
and  teacher 

o  The  passing  grade  for  testing  is  set  at  the  industry  apprentice  standard  of  70%  though 
this  may  be  revised  by  each  individual  teacher 

o  For  each  question  that  is  incorrectly  answered,  the  student  is  provided  the  specific 
location  in  the  Individual  Learning  Module  (ILM)  in  which  the  outcome  is  located, 
allowing  the  student  to  review  that  concept  prior  to  retaking  the  review  test 

o  Students  that  are  required  to  re-take  a  quiz/test  if  they  do  not  receive  a  score  of  70% 
receive  a  number  of  new  questions  (approximately  20%)  that  they  had  not  previously 
been  tested  on,  covering  specific  outcomes  of  that  particular  testing  module 

■  Some  assessments  are  up  to  60  questions  in  length 

■  Each  quiz  or  test  accessed  by  students  is  unique  (i.e.  two  students  taking  the 
same  test  at  the  same  time  will  have  different  questions) 
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■    All  questions  have  been  validated  by  RDC 
o    RDC  has  created  an  exam  bank  of  approximately  1800  questions  for  welding  at  the 
3400  level 

•  A  larger  practice  online  test  taken  prior  to  the  final  exam 

•  A  final  online  exam  for  each  course  that  must  be  supervised  by  a  teacher  at  a  designated 
writing  location  (i.e.  participating  school) 

•  Online  resources  as  uploaded  by  participating  teachers  or  by  RDC  into  the  central  resource 
repository  (i.e.  videos,  exemplars,  rubrics,  assignments,  etc.)  to  provide  a  library  of 
appropriate  resources 

o    Project  descriptions  can  be  adapted  and  placed  online 
o    Rubrics  for  common  projects  and  other  assessments 
o    Pictures  of  specific  welds  or  other  trade  skills 
o    Videos  of  weld  techniques  or  other  trade  skills 

•  TLM  is  primarily  designed  to  be  an  assessment  tool,  though  as  explained  above,  it  does 
possess  features  that  would  permit  teacher  and  student  access  to  specific,  appropriate  and 
approved  resources  uploaded  by  a  central  body. 

As  previously  indicated  TLM  in  this  pilot  is  formatted  to  the  new  3400  1st  period  apprenticeship 
carpentry  and  welding  courses  that  are  scheduled  to  be  implemented  in  September  2009.  This 
pilot  used  the  current  CTS  courses  that  are  either  aligned  or  supported  by  the  3400  level  courses 
and  these  courses  are  identified  in  Appendix  B  of  this  document.  This  pilot  is  examining  the 
TLM  model  for  online  assessment  and  is  not  assessing  the  3400  level  courses. 

Individual  Learning  Module  (ILMs)  Hardcopy  Resources 

ILMs  are  booklets  that  are  updated  and  supplied  annually  by  Apprenticeship,  Industry  and 
Training  (AIT),  Alberta  Advanced  Education  and  Technology  and  are  written  and  produced  for 
each  individual  module  and  revised  yearly  to  current  industry  standards  with  new  materials  and 
trade  practices  that  impact  that  specific  area.  ILM  users,  including  students  and  teachers,  are 
able  to  submit  any  suggestions  or  concerns  to  AIT  when  the  booklets  are  revised  each  year,  again 
ensuring  that  they  are  current  and  relevant. 

This  pilot  program  that  involves  RDC  and  Alberta  Education  offers  self-study  courses  online  that 
were  written  by  trade  content  experts  and  include  illustrated  learning  modules,  ILMs,  that  are 
supported  by  hardcopy  resources  provided  to  each  student.  In  2006,  RDC  through  Alberta 
Education  conducted  an  in-depth  review  of  some  of  these  resources  and  others  were  researched 
and  written  and  continue  to  be  developed  by  other  industry  groups  through  direction  and 
collaboration  with  AIT. 

TLM  offers  students  and  their  CTS  pilot  teachers  tools  for  development,  delivery,  reporting  and 
management  of  training  and  learning  materials.  It  uses  a  central  object  repository  to  facilitate  the 
creation,  storage  and  reuse  of  learning  content  that,  in  the  case  of  RDC  and  this  pilot,  are 
carpentry  and  welding  to  1st  period  apprenticeship.  Students  are  able  to  access  this  program 
through  any  computer  that  has  internet  access.  Students  participating  in  this  pilot  were  each 
supplied  with  the  appropriate  ILMs. 
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TLM  Technology 

This  pilot  operated  using  a  current  server  at  RDC.  However,  concerns  were  expressed  at  the  start 
of  the  pilot  related  to  the  abilities  of  the  server  to  handle  the  additional  requirements  of  the 
connection  to  the  five  school  sites,  the  approximate  thirty-four  students  and  seven  teachers.  Over 
the  course  of  the  pilot  only  one  technological  'glitch'  developed  related  to  TLM  and  that  was  an 
interruption  in  the  external  server  that  was  quickly  and  effectively  corrected. 

This  pilot  has  proven  that  the  external  server  was  capable  of  handling  the  small  number  of 
additional  users  and  indicated  that  the  RDC  system  was  well-developed.  It  is  anticipated  that  if  a 
greater  number  of  users  were  to  access  TLM  through  RDC  that  a  new  server  would  need  to  be 
purchased  as  had  been  discussed  during  the  negotiations  of  this  pilot  between  RDC  and  Alberta 
Education.  The  cost  of  an  appropriate  server  was  determined  to  be  approximately  $27,000  in 
late  2008  by  RDC  staff. 

Pilot  teachers  indicated  that  they  felt  comfortable  using  TLM  and  that  they  were  quickly  able  to 
navigate  through  the  online  features  and  understand  the  uses  and  opportunities. 

Any  user  questions  that  developed  in  the  early  stages  of  the  pilot  were  effectively  dealt  with 
through  a  telephone  contact  with  the  RDC  contact  technician  responsible  for  troubleshooting 
TLM.  Teachers  indicated  that  their  questions  met  with  an  immediate  and  effective  response  and 
that  their  students  did  not  experience  any  technology  concerns. 

Initial  teacher  training  on  the  use  of  TLM  consisted  of  one  in-service  session  lasting 
approximately  four  hours  at  RDC  in  a  computer  lab  and  a  significant  amount  of  this  time  was 
spent  by  the  teachers  'playing'  on  the  program  to  see  TLM  features  and  options. 

TLM  Site  Fees 

There  was  a  site  fee  charged  by  RDC  of  $4,000  per  school  to  permit  access  to  the  RDC 
developed  TLM  resource.  This  fee  was  a  reduction  from  the  usual  amount  charged  by  RDC  to 
Alberta  post-secondary  institutions  for  access  to  the  TLM  courses  that  they  have  developed. 

The  site  fees  and  other  pilot  costs  were  covered  by  Alberta  Education  through  a  conditional  grant 
to  Red  Deer  College. 

Alberta  Journeyman  Certification  Explanation 

Journeyman  Certificate  programs  combine  both  on-the-job  training  and  post-secondary  study  to 
prepare  graduates  to  work  as  tradespersons  in  Alberta.  Successful  completion  of  apprenticeship 
training  leads  to  a  Journeyman  Certificate.  There  are  up  to  five  levels  of  training,  also  called 
'periods',  that  must  be  taken  prior  to  gaining  eligibility  to  be  certified  as  a  journeyman  that 
depend  upon  the  trade  area  and  the  level  of  certification  desired.  To  better  explain  the  concept  it 
needs  to  be  understood  that  apprenticeship  is  the  process  of  training  and  journeyman  is  the 
certificate  earned. 
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To  move  from  one  level  or  period  of  apprenticeship  to  the  next  a  student  must2: 

•  Have  the  required  number  of  months  of  on-the-job  training 

•  Have  the  required  number  of  hours  of  on-the-job  training 

•  Successfully  complete  technical  training 

•  Pass  industry  exams  through  AIT 

•  Ensure  their  individual  Record  Book  is  updated  by  their  employer 

•  Send  their  Record  Book  to  AIT  for  a  Certificate  of  Progress  stamp 

TLM  uses  student  online  assessments  to  assist  high  school  students  meet  the  1st  period 
apprenticeship  technical  requirements  as  managed  by  AIT.  Once  certified  as  a  journeyman, 
many  trades  offer  graduates  the  opportunity  to  earn  a  Red  Seal,  which  is  recognized  in  other 
provinces  and  territories  that  participate  in  the  Red  Seal  program,  allowing  the  journeyman 
interprovincial  options  for  employment. 

Pilot  Schools  Background 

The  five  pilot  schools  involved  are  from  either  small  urban  or  small  rural  settings  and  are 
themselves  from  smaller  school  authorities.  The  subsequent  programs  that  are  offered  at  each 
school  are  delivered  through  specialized  labs  for  carpentry  or  welding  that  are  (1)  designed  and 
equipped  to  the  journeyman  level  as  determined  by  the  standards  developed  by  AIT,  or  (2)  are 
physically  smaller  general  labs  that  are  not  designed  nor  equipped  to  the  journeyman  level. 

The  type  of  CTS  lab  in  a  school  is  somewhat  dependent  upon  the  overall  student  registration 
count  of  the  school  and  subsequently  the  population  of  the  community  in  which  it  is  situated. 
The  student  registration  counts  at  each  pilot  school  are  provided  in  Chart  3  to  simply  indicate  the 
general  population  of  each  of  the  regular  program  high  schools  involved  in  the  pilot. 


2  Advanced  Education  and  Technology,  Post-Secondary  Education  System.  Website  http://aet.alberta.ca/post- 
secondary/credentials/definitions.aspx 
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Chart  3:  Student  Count 
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The  relative  student  population  of  each  pilot  school  is  a  general  indication  of  the  number  of 
potential  students  in  CTS  and  specifically  in  a  specialized  program  such  as  carpentry  or  welding 
as  examined  in  this  pilot.  It  may  be  possible  to  assume  that  the  larger  the  general  population  of 
the  school,  the  more  potential  students  may  be  registered  in  CTS  and  therefore  a  larger  number 
are  potentially  eligible  for  a  higher  level  specialized  1  st  period  apprenticeship  program.  There  is 
always  the  opportunity  for  an  engaging  CTS  teacher  in  a  small  school  program  to  have  a  larger 
percentage  of  students  registered  in  CTS  courses,  depending  upon  the  program  that  they  are  able 
to  offer. 


Charts  4  and  5  offer  an  indication  of  the  average  number  of  credits  earned  by  each  student  in  the 
four  pilot  schools  who  enrolled  in  welding  or  carpentry. 
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Chart  4:  Carpentry  Credit  Average 


3.0 


Average  Number  of  Carpentry  Credits  Earned  per  Enrolled 
Student  Granted  Credit 





2004 


2005 


2006 


2007 


2008 


Strathmore  HS 

Ecole  Secondaire  Lacombe  HS 
Provincial  Average 


Ponoka  Composite  HS 
Trochu  Valley  School 


Chart  5:  Welding  Credits  Average  
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The  average  number  of  earned  credits  in  carpentry  and  welding  is  below  the  threshold  of  five 
credits  per  student,  which  is  the  usual  target  that  high  schools  attempt  to  schedule  each  student  to 
earn  for  budgetary  reasons.  The  lower  number  of  earned  credits  in  carpentry  and  welding  in  the 
pilot  schools  and  as  a  provincial  average  may  be  due  to  a  couple  of  reasons: 
•  Some  CTS  modules  may  take  longer  to  complete  than  others  and  create  a  'time  crunch'  in  the 
allotted  student  timetable 
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Students  may  be  registering  in  modules  from  different  CTS  strands  not  solely  from  welding 
or  carpentry  within  the  course  of  a  semestered  CTS  course  block  and  this  is  not  reflected  in 
the  chart  data 


Further  evidence  of  this  is  provided  in  Chart  6  on  the  following  page  that  identifies  students  are 
taking  a  variety  of  strands  in  CTS  and  earning  more  credits  than  Charts  4  and  5  would  indicate. 

Chart  6:  Average  CTS  Credits     
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Chart  7  provides  a  count  for  the  total  number  of  students  registered  in  CTS  in  the  pilot  schools. 
Chart  7:  Number  of  Students  Registered  in  CTS 
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Pilot  Study  Question  Response 

Six  questions  presented  earlier  in  this  document  were  identified  as  the  focus  of  this  pilot  and  will 
be  addressed  in  order.  Information  will  be  provided  to  indicate  the  success  or  challenges  that 
were  evident  in  the  collection  of  data  for  each  area  and  the  possible  conclusions  determined 
(all  teacher  and  student  responses  are  provided  in  Appendix  D  of  this  study/ 

1.  Determine  the  impact  of  standardized  student  assessment  upon  opportunities  for  student 
success  that  potentially  narrows  the  gap  of  subjective  teacher-awarded  marks. 


As  the  observation  from  SAIT  instructors  presented  earlier  in  the  Background  section  of  this 
document  stated,  post-secondary  instructors  have  been  concerned  about  the  number  of  students 
that  they  see  in  their  courses  who  have  skills  below  what  their  high  school  CTS  marks  would 
suggest.  This  assertion  was  further  strengthened  by  an  observation  offered  by  a  CTS  teacher 
participating  in  this  pilot.  It  was  reported  by  an  administrator  at  this  CTS  teacher's  school  that  a 
number  of  students  from  other  schools  neighboring  the  community  were  transferring  into  this 
school  for  the  senior  year  in  order  to  access  additional  course  offerings  in  the  academic  core 
subjects  as  well  as  CTS.  The  CTS  marks  in  credits  previously  earned  by  the  transferring 
students  suggested  that  the  students  had  a  level  of  competence  and  success  that  was  relatively 
'significant'  as  indicated  by  their  awarded  transcript  marks  at  85%  and  higher. 

The  receiving  CTS  teacher  in  carpentry,  who  had  a  journeyman  certificate,  provided  this 
observation: 
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"Their  (the  student's)  previous  90%  is  the  equivalent  of  a  45%  in  my  program. ..they  just 
do  not  have  the  skills  that  they  are  supposed  to  if  they  had  finished  the  modules  that  their 
previous  school  said  that  they  did.  " 

One  high  school  student  who  is  registered  at  a  pilot  school  after  having  transferred  from  an  out- 
of-district  high  school  indicated  that  in  relation  to  the  course  marks  previously  received  in  CTS 
carpentry: 

"I  don 't  know  how  I  got  the  (high)  marks  that  I  did  at  my  last  school... I  really  didn 't  do 
anything  in  class. " 

TLM  offers  industry-standard  questions  and  resources  that  teachers  previously  did  not  have 
access  to.  The  standardized  online  assessment  permits  students  to  complete  exams  and  receive 
instant  marks  in  the  theoretical  components.  Students  must  complete  practical  applications  of 
their  skills  that  are  graded  by  the  classroom  teacher.  In  the  determination  of  the  final  overall 
course  grade  for  a  course,  the  classroom  teacher  determines  the  weighting  of  the  theoretical  and 
practical  components  to  determine  an  overall  course  grade. 

TLM  as  constructed  by  RDC  offers  the  opportunity  for  exemplars  and  rubrics  to  be  posted  to 
allow  each  teacher  and  student  access  to  industry  standard  resources  for  grading  of  practical 
assignments.  It  is  also  possible  to  develop  projects  and  post  them  on  TLM  permitting  teachers 
and  students  to  have  a  variety  of  projects  to  choose  from  as  an  additional  means  to  challenge 
both  students  and  teachers. 

Each  CTS  teacher  and  school  administrator  participating  in  this  pilot  specifically  stated  when 
asked,  that  online  assessment  would  strengthen  CTS  by  helping  to  ensure  that  students  were 
accurately  graded. 

The  principal  of  the  participating  outreach  school  indicated: 

"Standardizing  student  assessment  would  be  of  a  great  value.  As  we  are  an  alternative 
school,  we  get  students  from  many  different  schools.  Having  standardized  modules,  tests, 
etc.  would  be  of  a  great  value.  It  would  definitely  level  the  playing  field.  On  the  other  hand, 
sometimes  our  programs  are  seen  as  being  easier  for  the  student  to  achieve  success.  This 
would  also  do  away  with  this  view. 

And  concludes: 

"Therefore,  TLM  acts  as  a  level  playing  field  for  theory  delivery.  There  is  no  question  as  to 
the  student's  performance  level.  There  then  is  no  question  as  to  whether  an  outreach 
program  has  adjusted  the  required  level  of  student  performance  -  which  is  in  question  from 
schools  who  identify  themselves  as  academic  schools.  We  are  then  respected  for  adjusting 
only  the  learning  strategies.  " 

2.  Determine  the  value  of  this  online  assessment  tool  for  potential  use  by  non-journeyman 
CTS  teachers. 

This  question  was  posed  to  the  participating  pilot  CTS  teachers  and  their  general  responses 
indicate  that  the  TLM  program  is: 
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•  A  strong  resource  for  assessment  and  as  a  repository  for  current  industry  standard 
material 

•  A  resource  that  cannot  be  purchased  from  any  commercial  sources  (i.e.  publishers)  that 
create  material  for  other  subject  areas  such  as  Math  or  Science 

•  Provides  opportunities  to  challenge  students  and  their  teachers 

o  Encourages  non-journeyman  teachers  to  learn  new  skills  and  stay  ahead  of  their 
students  through  appropriate  professional  learning  opportunities  that  may  then  be 
reflected  through  improved  assessment  strategies 

It  is  noted  that  this  resource  is  intended  to  assist  teachers  in  the  recognition  that  non-journeyman 
CTS  teachers  may  not  have  the  benefit  of  the  practical  experience  and  theoretical  knowledge  that 
journeyman  may  possess.  TLM  offers  assistance  for  the  theoretical  knowledge  and  may  provide 
visual  resources  that  are  downloaded  into  the  program  central  repository.  These  resources  would 
provide  practical  guides  for  students  and  teachers  such  as:  exemplars,  rubrics,  videos,  common 
projects,  industry  web  resources  and  teacher  developed  resources. 

As  stated  earlier  in  this  document,  non-journeyman  CTS  teachers  are  not  permitted  by  AIT  to 
teach  the  1st  period  apprenticeship  program  without  the  support  of  a  journeyman  instructor  and 
have  their  students  then  positioned  to  challenge  the  1st  period  exams.  This  supporting 
arrangement  is  left  to  the  individual  school  or  school  authority  to  determine  through  a 
partnership  with  an  outside  organization  or  the  hiring  of  a  journeyman  who  would  then  instruct 
under  the  supervision  of  a  school  or  school  authority  certificated  staff  member. 

One  possible  concern  raised  with  the  use  of  non-journeyman  teachers  providing  instruction  to  the 
1st  period  apprenticeship  level,  even  with  the  support  of  a  journeyman  instructor,  is  that  non- 
journeyman  may  not  have  the  practical  experience  in  their  field  that  is  considered  to  be  a  strong 
and  vital  benefit  by  many  in  the  field  when  instructing  students  in  welding  or  carpentry  and  other 
trades.  They  need  to  be  able  to  'stay  ahead'  of  student  theoretical  learning,  however  they  are 
unable  to  provide  instruction  from  an  advanced  practical  background.  This  is  an  indication  that  a 
school  or  school  authority  partnership  with  a  journeyman  in  a  supporting  instructional  role  is 
central  to  the  overall  student  and  program  success. 

To  maintain  the  standards  and  rigor  of  the  new  revisions  to  the  CTS  program,  a  journeyman  is 
required  to  provide  instruction  to  those  students  who  are  preparing  for  1st  period  apprenticeship. 
The  standards  and  rigor  are  the  same  as  those  offered  in  post-secondary  institutions.  Anything 
less  would  diminish  the  intent  of  the  CTS  apprenticeship  program.  Those  students  who  do  not 
intend  to  enter  into  an  apprenticeship  but  would  like  to  take  these  courses  may  do  so  and  receive 
instruction  from  an  instructor  without  journeyman  status.  Those  schools  and  school  authorities 
that  would  like  to  offer  the  apprenticeship  program  and  do  not  have  a  journeyman  may  enter  into 
an  agreement  with  a  journeyman  to  teach  the  entire  program  under  the  supervision  of  a 
classroom  teacher. 

The  potential  exists  to  expand  TLM  to  other  CTS  areas  that  are  not  at  the  3400  level,  though  it  is 
recognized  that  this  would  be  a  labor  intensive  process.  Participating  pilot  teachers  indicate  that 
the  use  of  TLM  would  have  a  significant  and  positive  impact  upon  addressing  the  concerns 
evident  with  the  teacher-awarded  mark  gap,  as  was  observed  in  the  previous  Question  1 
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response.  The  vast  majority  of  credits  earned  by  students  in  CTS  are  at  the  introductory, 
intermediate  and  advanced  level  that  are  not  equivalent  to  the  new  3400  level. 

3.  Assess  the  impact  of  this  online  assessment  tool  upon  student  achievement. 

The  scope  of  this  pilot  did  not  permit  a  thorough  examination  of  the  nature  of  the  work  required 
to  adequately  address  how  TLM,  as  an  online  standardized  assessment  tool,  may  impact  student 
achievement  in  CTS.  The  small  number  of  schools,  teachers  and  students  and  the  fact  that  this 
pilot  was  only  conducted  over  the  course  of  one  semester  are  the  contributing  factors  to  the 
inability  to  fully  examine  this  concept.  However,  there  were  a  number  of  anecdotal  comments 
provided  by  participating  teachers  that  offered  their  perception  of  TLM  that  may  assist  by 
providing  a  limited  view. 

Teachers  participating  in  the  pilot  indicated  their  belief  that  TLM  would  challenge  students  to 
accomplish  more  in  the  theoretical  area  of  carpentry  and  welding.  The  exposure  to  industry 
standard  resources  was  indicated  to  be  a  significant  "carrot"  for  many  students  and  that  many 
appreciated  the  connection  to  "real-world  scenarios". 

It  was  observed  by  pilot  participants  that  the  TLM  program  may  be  beneficial  to  other 
educational  programs  such  as  the  Registered  Apprenticeship  Program  (RAP)  and  to  Work 
Experience  placements.  The  opportunity  to  place  students  with  appropriate  businesses  may 
improve  in  some  situations. 

"I  see  that  TLMs  can  help  RAP/work  experience  coordinators  relate  to  people  in  our 
work  placements.  There  is  the  TLM  theory  which  can  assist  the  students  to  understand 
well  enough  to  ask  question  of  their  employer/journeymen.  It  also  outlines  the  scope  of 
expectations  that  are  needed.  For  small,  rural  communities — with  serious  thought  and 
development,  the  linkage  between  RAP  and  TLMs  may  improve  a  partnership  between 
those  who  need  to  run  their  business  and  the  school  who  wants  students  to  have  a  more 
rounded  perspective  that  the  business  is  prepared  to  offer.  After  all,  businesses  are  run 
for  the  purpose  of  making  a  living  and  not  for  educating  our  students.  " 

In  the  experience  of  the  outreach  program,  student  participation  in  TLM  was  used  as  the 
incentive  to  complete  other  curricular  requirements  for  a  "bright"  student  who  was  not 
challenged  by  "regular"  courses.  This  particular  student  did  successfully  complete  the  other 
requirements  and  progressed  in  TLM. 

It  was  agreed  upon  by  the  participating  teachers  that  the  students  who  have  the  opportunity  to 
participate  in  a  program  such  as  this  will  achieve  to  a  higher  standard  than  in  a  regular  CTS 
program.  The  emphasis  upon  industry  standards  with  a  minimum  mark  of  70%  and  the  specific 
AIT  requirements  for  completion  of  the  multiple  1st  period  apprenticeship  journeyman 
components  assists  to  ensure  this. 

It  was  stated  by  a  participating  teacher  and  echoed  in  separate  responses  by  other  participants 
that: 

"If  we  set  the  bar  at  a  high  level  for  the  students  and  show  them  what  the  real-world 
expects,  they  will  rise  to  that  level.  I  have  seen  it  already  with  my  students  and  we  have 


19 


younger  students  who  can 't  wait  to  get  involved  with  this  and  show  what  they  are  capable 
of  achieving.  " 

As  previously  indicated  in  this  document,  one  student  completed  all  of  the  theoretical 
components  of  the  1  st  period  apprenticeship  program  and  requested  the  opportunity  to  challenge 
the  AIT  1st  period  apprenticeship  Journeyman  Exam  in  welding.  A  second  student  was  working 
to  complete  the  last  module  and  would  also  be  in  position  to  challenge  the  AIT  exam. 

A  stumbling  block  in  the  process  was  concern  expressed  by  AIT  that  the  student's  high  school 
CTS  teacher  was  not  certified  as  a  journeyman.  This  CTS  teacher  was  supported  by  a  certified 
journeyman  welder  from  RDC  to  assist  in  facilitating  this  pilot  on  a  part-time  basis.  AIT's 
position  was  that  complete  instruction  needed  to  be  provided  by  a  certified  instructor. 

The  fact  that  a  student  was  able  to  complete  the  requirements  and  be  placed  in  a  position  to  write 
the  AIT  exam  is  significant.  Discussions  between  RDC  and  AIT  may  permit  the  student  in 
question  to  write  the  exam  through  the  approval  of  AIT,  if  RDC  signs-off  on  all  the  required 
skills  that  the  student  in  question  needs  to  demonstrate.  RDC  has  since  completed  the  sign-off. 

4.  Assessing  the  impact  upon  student  and  teacher  perception  of  the  online  tool. 

Teachers  and  students  were  impressed  with  the  robust  nature  of  the  program.  Teachers,  whether 
they  were  journeyman  or  non-journeyman,  agreed  that  TLM  challenged  both  them  and  their 
students  in  a  positive  manner  in  which  they  previously  had  not  experienced. 

Their  joint  observations  regarding  TLM  and  online  student  assessment  include: 

•  The  ability  to  access  current  industry- standard  material  from  one  site  that  is  pre-made  has 
not  been  possible  before  this 

•  The  ease  of  the  online  program  operation  was  noted  as  a  definite  strength 

•  This  online  program  has  the  potential  to  expand  and  address  issues  and  concerns  with  the 
delivery  and  marking  of  other  CTS  courses  that  are  at  the  non-3400  level  and  at  the 
introductory,  intermediate  and  advanced  levels  as  well 

•  Teachers  feel  that  the  exposure  to  this  advanced  technical  learning  and  the  standardized 
assessment  has  caused  other  teachers  to  revise  their  previous  perception  of  CTS.  One 
participating  teacher  relayed  a  colleague's  comment,  "If  it  has  a  standardized  test  like  a 
diploma,  then  it  must  be  an  important  class.  " 

•  "The  difficulty  level  gives  credibility  to  the  vocation  and  makes  students  realize  just  how 
much  an  apprentice  welder  is  expected  to  know.  " 

•  Students  stated  their  belief  that  a  standardized  and  supervised  exam,  such  as  those  found 
at  the  conclusion  of  each  course,  increased  the  "value"  and  the  overall  "status"  of  CTS 
increased,  "This  is  not  easy  stuff.  You  have  to  study  and  know  it  or  you  will  not  pass. 
The  online  tests  are  tough  and  it  is  an  exam  that  means  a  lot.  " 

•  Student  inquiries  into  the  possibility  of  continuing  to  access  TLM  and  further  their 
training  over  the  summer  months  outside  of  the  regular  school  calendar  year 

•  The  opportunity  for  TLM  to  serve  as  a  central  repository  for  resources  that  teachers  and 
students  can  access  provides  a  'level  playing  field'  that  assists  to  ensure  that  the  latest 
industry- standard  material  is  available. 
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o  This  feature  needs  to  be  explored  more  and  would  greatly  assist  both  journeyman 
instructors  who  may  have  some  background  materials  and  the  non-journeyman 
instructors  who  have  to  'scramble'  to  collect  resources  that  are  current. 

Administrators  indicated  that  TLM  provides  the  opportunity  to  improve  the  status  of  CTS  due  to 
standardizing  assessments  and  the  necessity  for  students  to  be  better  prepared  to  meet  the  course 
requirements.  It  was  indicated  that  this  program  will  also  require  some  students,  teachers,  school 
counselors,  parents  and  school  administrators  to  re-think  their  view  of  CTS  with  the  realization 
that,  "This  program  is  tough  and  not  an  easy  credit.  Students  have  to  work  and  know  their 
stuff... for  some  it  will  be  a  rude  awakening.  " 

5.  Determining  the  potential  for  future  expansion  of  this  tool  beyond  the  initial  pilot  schools. 

Participating  pilot  teachers  indicated  their  belief  that  TLM  displayed  strong  characteristics  that 
would  make  it  a  desirable  resource  to  make  available  to  all  high  schools  in  Alberta. 

There  were  a  number  of  positive  factors  that  they  identified  to  support  their  assertions: 

•  TLM  presents  a  significant  opportunity  to  encourage  non-journeyman  and  journeyman 
CTS  teachers  to  improve  upon  their  skills 

•  The  use  of  current  and  relevant  industry-based  information  is  a  resource  that  is  not 
available  from  any  other  source 

•  There  is  a  significant  opportunity  to  expand  TLM  beyond  carpentry  and  welding  to  all 
other  CTS  courses  that  have  journeyman  credentialing 

•  The  opportunity  to  use  TLM  in  welding  at  the  1  st  period  apprenticeship  was  the  incentive 
to  continue  in  high  school  for  one  student  (an  observation  shared  by  an  RDC  instructor) 

•  Younger  high  school  students  are  enthused  about  the  opportunity  to  use  TLM  once  they 
enter  Grade  1 1  and  have  completed  the  required  introductory  and  intermediate  course  work 

•  The  "real- world"  learning  for  students  was  significant  and  'invaluable' 

•  A  belief  that  TLM  could  be  beneficial  to  all  areas  of  CTS  at  the  introductory,  intermediate 
and  advanced  levels,  not  just  for  certain  3400  level  courses 

•  Opportunity  to  address  the  school-awarded  mark  gap  that  has  been  experienced  by  CTS 
teachers  in  the  pilot  schools 

•  If  there  was  a  TLM  type  system,  which  included  the  theoretical  material,  available  to  all 
CTS  teachers  in  the  province,  this  would  improve  the  instruction  and  assessment  of  the 
modules 

•  One  student  in  carpentry  has  requested  that  he  be  able  to  access  TLM  over  the  summer  to 
permit  him  to  complete  additional  courses 

•  Opportunity  for  creative  partnerships  between  schools,  school  authorities,  businesses  and 
post-secondary  institutions  through  TLM  would  improve  relationships  and  community 
awareness  for  school  programming 

•  The  effort  that  Red  Deer  College  personnel  have  put  into  making  TLM  an  effective  online 
tool  for  carpentry  and  welding  that  is  supported  by  strong  professional  learning 
opportunities  through  their  summer  program  is  a  model  that  other  institutions  should  learn 
from 
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•  The  collaboration  and  support  that  Red  Deer  College  has  initiated  and  developed  with 
schools  and  school  authorities  needs  to  be  replicated  in  other  areas  of  the  province  with 
other  post-secondary  institutions  and  school  authorities 

The  positive  factors  stated  by  the  participating  teachers  also  included  a  number  of  caveats 
regarding  the  potential  to  expand  this  resource  availability: 

•  Schools  using  TLM  need  to  have  their  lab  equipped  to  the  level  required  to  teach  to  the 
journeyman  level  or  access  is  needed  to  a  facility  that  is  at  the  required  level 

•  Schools  need  to  have  computer  access  in  the  CTS  labs  for  student  and  teacher  interaction 
when  using  TLM 

•  Non-journeyman  CTS  teachers  may  face  a  significant  challenge  to  'stay  ahead'  of  the 
learning  of  their  students  as  they  do  not  have  the  practical  experience  of  journeyman 

•  The  costs  associated  with  TLM  site  fees  and  offering  1st  period  apprenticeship 
programming  at  the  high  school  level  may  be  cost-prohibitive  for  small  schools  or  school 
authorities 

•  Strong  and  practical  professional  learning  opportunities  need  to  be  made  available  for  CTS 
teachers  similar  to  the  offerings  available  at  Red  Deer  College  in  the  summer  program  to 
better  equip  teachers  to  1  st  period  apprenticeship  status 

•  Schools  and  school  authorities  examining  the  potential  to  create  partnerships  with  outside 
organizations  or  post-secondary  institutions  to  share  journeyman  instructors  may  face 
significant  challenges  in  small,  rural  communities  due  to  the  lack  of  availability  of 
appropriate  and  qualified  journeyman  willing  and  able  to  share  their  expertise 

•  Strong  connections  between  the  schools  and  post-secondary  institutions  (Red  Deer  College 
in  this  instance)  need  to  be  built  and  maintained  to  ensure  that  teachers  are  effectively  using 
TLM 

•  Connections  need  to  be  built  between  schools  within  school  authorities  and  neighbouring 
school  authorities  in  order  to  more  efficiently  access  programs,  facilities,  resources  and 
personnel 

Expansion  of  this  online  standardized  student  assessment  resource  to  other  high  schools  in  Alberta 
that  will  be  providing  the  3400  level  courses  particularly  in  carpentry  and  welding  and 
opportunities  for  schools  and  school  authorities  to  maintain  current  CTS  programs  or  expand  upon 
them  needs  to  consider  a  number  of  factors: 

>  The  ability  of  Red  Deer  College  to  expand  the  number  of  schools  and  students  that  access 
the  program  that  they  have  developed  given  the  constraints  of: 

o   The  college  personnel  and  their  time  and  other  professional  responsibilities 
o   The  college  budget 

o   The  college  mandate  to  serve  Central  Alberta 

>  The  opportunity  for  Alberta  Education  to  undertake  a  leadership  role  in  the  assessment  of 
CTS  within  the  constraints  of: 

o   The  Ministry  vision  to  develop  a  role  in  CTS  standardized  assessment 
o    Ministry  personnel  considerations 
o    Ministry  budget  considerations 

>  The  desire  for  school  authorities  and  schools  outside  of  the  current  pilot  to  embrace  an  area 
of  standardized  student  assessment  that  has  not  previously  been  explored  (i.e.  CTS) 
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>  The  desire  of  schools  and  school  authorities  to  discuss  partnerships  or  sharing 
arrangements  with  other  schools,  school  authorities  and  post-secondary  institutions 

>  The  desire  of  schools  and  school  authorities  to  maintain  current  CTS  programs  or  expand 
them  to  better  meet  the  needs  of  students  and  the  community 

6.  Determining  the  value  of  specific  student  achievement  measures  in  the  Student 
Assessment  Framework. 

As  part  of  the  CTS  Initiative,  the  System  Improvement  Group  has  been  tasked  with  developing  a 
CTS  Student  Assessment  Framework  that  is  intended  to  accomplish  two  tasks:  (1 )  measure  the 
impact  of  the  delivery  and  implementation  of  the  revised  program  and  (2)  summative 
measurement  of  student  achievement  in  CTS.  The  framework  would  assist  in  meeting 
challenges  presented  by  the  information  gap  in  a  critical  subject  area  for  high  school  students  and 
better  position  educators  to  address  the  needs  of  individual  students  in  CTS.  The  current  draft  of 
this  framework  is  provided  in  Appendix  E  of  this  document  and  it  should  be  noted  that  the 
framework  is  part  of  a  larger  document,  Career  and  Technical  Studies  (CTS)  Student  Assessment 
Framework:  Focus  Upon  Two  Tasks,  Draft  Document  12.1  posted  on  the  CTS  Education  Base 
website  internal  to  Alberta  Education  and  that  this  document  is  currently  undergoing  revision. 

In  CTS  there  are  several  key  questions  regarding  the  development  and  use  of  student  assessment 
measures  that  are  not  currently  asked  of  schools  and  school  authorities  in  the  development  of 
education  plans,  yet  are  critical  and  are  directing  the  efforts  of  many  stakeholders: 

•  What  is  the  minimum  that  we  want  students  to  have? 

•  What  should  students  in  CTS  know,  be  able  to  do,  and  value? 

•  Have  the  graduates  who  have  participated  in  CTS  acquired  the  intended  learning? 

•  What  are  the  contributions  of  the  school  and  the  CTS  program  to  student  growth? 

•  How  can  student  learning  be  improved? 

•  What  should  students  be  learning  and  in  what  ways  should  they  be  growing? 

•  What  are  students  actually  learning  and  in  what  ways  are  they  actually  growing? 

•  What  should  we  be  doing  to  facilitate  student  learning  and  growth? 

•  How  should  we  measure  student  learning  and  growth? 

The  construct  of  a  student  assessment  framework  for  CTS  will  assist  educators  to  better  meet  the 
needs  of  individual  students  by  providing  data  to  educators,  allowing  program  delivery  revisions 
to  be  made  at  the  classroom,  school  authority  and  provincial  levels  in  a  timely  manner. 

The  student  assessment  framework  in  CTS  is  an  opportunity  to  build  upon  the  successes  that  are 
presented  with  the  considerable  work  done  by  all  stakeholders  in  revising  the  high  school  CTS 
program.  The  timely  collection  of  data  will  benefit  schools,  school  authorities,  post-secondary, 
industry,  the  province  and  ultimately  the  students. 

The  framework  was  developed  initially  through  focus  groups  held  with  the  diverse  membership 
that  comprises  the  Career  and  Technology  Studies  Advisory  Committee  and  was  then  reviewed 
through  presentations  and  feedback  received  from  a  number  of  stakeholder  groups.  These 
stakeholder  groups  included: 
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•  Career  and  Technology  Studies  Advisory  Committee 

•  Innovative  Learning  Services  Unit  (ILS),  Calgary  Board  of  Education  (CBE) 

•  CTS  teachers  from  the  CBE  and  Calgary  Catholic  through  ILS 

•  University  of  Calgary,  Faculty  of  Education  Professor 

•  College  of  Alberta  School  Superintendents  (CASS)  Executive 

•  Southern  Alberta  Institute  of  Technology  Department  Heads  and  Technical  Instructors 

•  Red  Deer  College,  Regional  Trades  Initiative 

•  CTS  Project  Management  Team  (internal  to  Alberta  Education) 

There  are  currently  twenty-nine  measures  proposed  to  analyze  the  impact  of  the  delivery  and 
implementation  of  the  Revised  CTS  Program  and  thirteen  measures  for  the  assessment  of  student 
achievement.  This  pilot  conducted  with  RDC  examined  one  of  the  measures  of  student 
achievement: 


The  number  of  students  who  access  an  online  test  provided  by  the  Regional  Trades  Initiative 
in  carpentry  or  welding  following  the  completion  of  a  grouping  of  courses 


TLM  allows  the  opportunity  to  assess  student  abilities  in  a  specific  area  and  provide  an 
indication  of  how  the  introductory,  intermediate  and  advanced  CTS  courses  are  preparing 
students  to  take  the  3400  level  1st  period  apprenticeship  program. 

This  online  program  would  assist  in  addressing  the  school-awarded  mark  gap  identified  as  a 
concern  by  the  instructors  from  SAIT  and  by  pilot  school  CTS  teachers. 

Current  Status  of  TLM  at  Red  Deer  College 

Over  the  course  of  the  last  several  months,  additional  interest  in  the  use  of  TLM  has  been 
expressed  by  a  variety  of  education  stakeholders  through  their  participation  in  this  pilot  or 
through  professional  learning  opportunities  that  RDC  has  provided.  The  TLM  opportunities 
include: 

•  RDC  expanding  TLM  to  include  1 st  period  apprenticeship  mechanics 

•  RDC  using  TLM  to  train  high  school  teachers  in  welding,  carpentry  and  mechanics 

•  Two  principals  in  Golden  Hills  Regional  Division  are  receiving  their   1st  period 
apprenticeship  technical  teacher  training  in  welding  instruction  through  TLM 

o  Eight  weeks  of  practical  1st  period  apprenticeship  training  is  being  condensed  into  six 
weeks  of  teacher  training  offered  in  one  week  courses  each  summer  at  RDC  in 
welding,  carpentry  and  mechanics 

o  These  courses  are  in  the  process  of  being  recognized  as  transfer  credits  to  the  Faculty 
of  Education  at  the  University  of  Alberta 

•  One  principal  in  St.  Thomas  Aquinas  Regional  School  Division  is  receiving  his  1st  period 
apprenticeship  welding  instruction  through  TLM 

•  Hutterite  colonies  in  Golden  Hills  Regional  Division  have  expressed  interest  in  accessing 
3400  level  welding  course  instruction  for  their  students 

•  SunChild  First  Nations  have  expressed  interest  in  welding  instruction  for  their  students 

•  Home  education  advocates  are  exploring  this  resource  for  welding  instruction 
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One  Math  teacher  in  Lacombe,  who  observed  the  use  of  TLM  in  the  CTS  pilot  at  that 
participating  school,  has  investigated  the  resource  and  has  expressed  an  interest  in  building  an 
application  for  Lacombe  High  School  using  the  Math  curriculum. 

Some  of  the  schools  that  have  participated  in  the  pilot  have  expressed  an  interest  in  continuing 
use  of  TLM  in  September  2009  regardless  of  the  findings  of  the  pilot  study  or  financial  support 
for  the  site  fees  from  Alberta  Education. 

Conclusion 

This  was  a  pilot  that  centered  on  a  limited  number  of  schools,  teachers,  students  and  time  and 
was  not  as  in-depth  a  study  as  would  have  been  preferred  under  ideal  circumstances. 
Nonetheless,  there  are  components  of  TLM  that  were  explored  and  identified  in  this  pilot  as 
possessing  the  ability  to  significantly  benefit  students  and  teachers  in  CTS.  The  opportunity  to 
standardize  student  assessment  on  the  theoretical  component  of  skills  attainment  and  reduce  the 
mark  gap  between  programs,  thereby  providing  a  better  indication  of  student  abilities  and  a 
means  to  resource  current  industry-standard  resources  is  a  tool  that  would  assist  students, 
teachers,  school  authorities,  employers  and  post-secondary  institutions. 

It  appears  that  Red  Deer  College  has  spent  considerable  time  and  effort  in  developing  TLM  for 
the  3400  level  courses  and  that  the  online  tool  has  performed  better  than  anticipated.  At  the  start 
of  this  pilot,  technical  issues  were  anticipated  however  they  did  not  materialize  at  all.  Pilot 
teachers  did  not  report  any  technical  concerns  other  than  one  server  issue  two  weeks  into  the 
pilot  that  was  quickly  and  effectively  handled.  It  should  be  noted  that  of  the  seven  participating 
pilot  teachers,  not  all  would  consider  themselves  to  be  'technologically  savvy'  and  two  of  them 
initially  indicated  that  they  could  be  referred  to  as  'technology  illiterates'. 

The  four-hour  professional  learning  opportunity  provided  by  RDC  at  the  start  of  the  pilot  and 
their  follow-up  contact  with  the  pilot  teachers  worked  to  effectively  answer  any  questions  and 
provide  the  knowledge  necessary  to  navigate  the  system.  All  teachers  reported  that  after  a  short 
time  using  TLM  that  they  felt  comfortable  and  confident  using  it,  though  one  teacher  required 
additional  support. 

The  opportunity  that  TLM  provides  to  manage  available  resources  as  a  central  repository  and  to 
provide  standardized  student  assessment  is  a  significant  resource  that  may  assist  the  further 
development  of  CTS  across  the  province,  particularly  in  smaller  centers  that  do  not  have  access 
to  specialized  personnel  or  materials.  With  the  continued  challenge  that  many  schools  and 
school  authorities  face  in  attracting  and  retaining  qualified  and  skilled  CTS  teachers,  TLM  would 
assist  in  two  significant  areas  of  classroom  instruction,  that  of  finding  appropriate  and  current 
resources  and  in  the  creation  of  appropriate  and  standardized  student  assessments. 

This  pilot  was  viewed  as  a  work  'add-on'  by  the  participating  schools  and  teachers  offered  TLM 
in  addition  to  their  existing  courses.  Some  teachers  were  better  equipped  and  prepared  to  offer 
TLM  and  the  results  indicated  this.  Regardless  of  the  credits  earned  by  students  and  the 
challenges  experienced,  all  teachers  and  students  agreed  that  TLM  is  a  strong  value-added 
resource  that  should  be  expanded  to  other  CTS  courses. 
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To  fully  develop  TLM,  schools  and  school  authorities  need  to  explore  partnership  opportunities 
with  other  school  authorities,  post-secondary  institutions  and  industry  that  would  allow  other 
stakeholders  to  play  a  role  in  the  development  of  CTS  programs.  The  sharing  of  responsibilities 
and  resources  would  better  position  students,  teachers,  parents,  schools,  school  authorities  and 
the  greater  community  in  meeting  the  changing  and  diverse  needs  of  each  community. 

Red  Deer  College  has  created  a  strong  Alberta  CTS  resource  that  appears  to  be  accepted  by  the 
field.  The  determination  of  some  schools  who  did  participate  in  the  pilot  to  continue  use  of  TLM 
after  the  completion  of  the  pilot  and  the  interest  shown  in  TLM  by  other  stakeholders  is  a 
possible  indication  of  its  viability  and  acceptance. 

Recommendations 

1.  That  Alberta  Education  pursues  a  partnership  with  Red  Deer  College  to  expand  use  of 
The  Learning  Manager  online  student  assessment  program  into  a  larger  pilot 
opportunity  with  additional  high  schools. 

Rationale:  The  original  pilot  consisted  of  a  small  number  of  schools  and  students  and  produced 
strong  results  and  additional  interest  in  the  online  resource.  A  larger  pilot  that  is  presented  over 
the  time  of  a  full  school  year  with  additional  upfront  work  may  provide  additional  data  that 
would  assist  in  the  determination  of  other  opportunities  for  online  standardized  student 
assessment  in  CTS  and  the  potential  for  more  wide-spread  student  access  to  TLM. 

The  introduction  of  the  3400  level  CTS  courses  in  TMT  and  BIT  in  September  2009  would  allow 
the  opportunity  for  a  larger  TLM  pilot  to  proceed  and  not  be  viewed  as  an  'add-on'  by  pilot 
teachers  as  there  would  not  be  any  need  to  map  the  'old'  CTS  Program  of  Studies  to  the  new 
curriculum. 

a.  That  Red  Deer  College  examine  opportunities  to  provide  The  Learning  Manager 
to  school  authorities  in  Alberta  through  agreements  as  a  tool  to  provide 
additional  resources  and  standardized  assessment  for  CTS  teachers  at  the  3400 
level  of  programming. 

Rationale:  It  has  been  indicated  that  some  pilot  schools  were  going  to  pursue  use  of  TLM 
beyond  this  pilot  and  RDC  was  gathering  information  for  them  in  order  to  accommodate  this. 
Schools  and  school  authorities  outside  of  this  pilot  may  be  interested  in  exploring  this 
opportunity  if  they  are  presented  with  the  information  and  professional  learning. 

2.  That  Red  Deer  College  and/or  other  post-secondary  institutions  continues  work  to 
expand  TLM  beyond  the  two  trades  areas  of  carpentry  and  welding  examined  in  this 
pilot  into  other  1st  period  apprenticeship  programs. 

Rationale:  The  opportunities  that  have  developed  in  carpentry  and  welding  as  part  of  this  pilot 
have  indicated  that  there  is  the  potential  for  this  resource  to  assist  in  other  trade  areas. 
Mechanics  was  recently  developed,  though  not  tested  in  this  pilot  and  is  in  the  introductory 
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stages  of  use  to  train  teachers.  There  are  other  trade  areas  that  would  benefit  from  the  adaptation 
of  TLM  for  assessment  and  resources. 

3.  That  Alberta  Education  and  Red  Deer  College  examine  the  opportunity  to  expand  The 
Learning  Manager  online  student  assessment  program  or  other  online  student 
assessment  program  to  CTS  courses  at  the  introductory,  intermediate  and  advanced 
levels. 

Rationale:  This  resource  displays  characteristics  that  would  assist  in  addressing  concerns 
identified  by  post-secondary  instructors  and  high  school  CTS  teachers  regarding  school-awarded 
marks  and  actual  student  skill  level  that  may  not  be  reflective  of  course  marks.  Many  students 
will  not  take  the  advanced  3400  level  courses  but  may  take  other  levels  of  CTS  and  apply  those 
earned  credits  towards  admission  to  post-secondary  programs  or  to  industry.  Standardized 
student  assessment  of  the  theoretical  component  of  the  course  work  would  assist  to  ensure  that 
the  mark  a  student  earns  is  indicative  of  their  actual  ability,  no  matter  which  high  school  program 
in  Alberta  that  they  were  registered  in  and  assist  to  address  the  variety  of  training,  experience 
and  certification  that  CTS  teachers  possess.  The  central  repository  of  materials  in  an  online 
program  such  as  TLM  would  also  assist  CTS  teachers  to  find  and  use  appropriate  resources. 

4.  That  Alberta  school  authorities  be  actively  encouraged  by  Alberta  Education  to  explore 
partnerships  and  shared  facility  agreements  with  neighboring  school  authorities  and 
post-secondary  institutions  to  more  effectively  utilize  programs,  facilities,  resources  and 
personnel. 

Rationale:  Many  schools  and  school  authorities  have  reported  challenges  in  recruiting  and 
retaining  qualified  CTS  teachers  and  the  facility  and  resource  requirements  to  offer  many 
programs  are  additional  financial  concerns.  Opportunities  to  'share'  students,  teachers  and 
facilities  needs  to  be  openly  and  constructively  discussed  by  school  authorities  in  order  to  best 
meet  the  needs  of  individual  students  and  Alberta  Education  has  the  opportunity  to  undertake  a 
leadership  role  in  this  process. 

The  current  AIT  requirement  that  students  who  wish  to  challenge  the  1st  period  journeyman 
exam  receive  their  instruction  from  a  certificated  journeyman  will  require  schools  and  school 
authorities  to  creatively  explore  partnership  opportunities. 

5.  That  Alberta  Education  continue  dialogue  with  Apprenticeship,  Industry  and  Training 
(AIT),  Alberta  Advanced  Education  and  Technology  to  address  the  status  of  students 
who  complete  the  3400  level  courses  through  the  instruction  offered  by  a  non- 
journeyman  CTS  teacher  and  their  ability  to  write  1st  period  journeyman  exams. 

Rationale:  Access  to  journeyman  CTS  teachers  is  a  challenge  that  many  schools  and  school 
authorities  are  facing  and  this  may  affect  opportunities  for  some  students.  Additional 
alternatives  to  permit  the  'sign-off  on  the  skills  attained  and  the  displayed  knowledge  of 
students  by  journeyman  instructors  who  were  satisfied  that  students  possess  the  necessary 
abilities  would  then  potentially  permit  more  students  to  access  this  AIT  opportunity.  The  clarity 
of  information  provided  to  students  and  teachers  on  the  opportunity  to  write  AIT  exams  or  the 
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guidelines  around  them  will  have  an  impact  upon  the  number  of  students  who  register  for  these 
3400  level  courses. 

This  needs  to  be  explored  in  order  to  mitigate  any  concerns  regarding  the  standards  of  instruction 
that  AIT  may  be  concerned  students  are  receiving  and  not  diminish  the  status  of  journeyman 
certification  in  Alberta. 

6.  That  Alberta  Education  explores  the  opportunity  with  Red  Deer  College  to  assume  a 
central  leadership  role  in  the  future  use  of  The  Learning  Manager  online  student 
assessment  program  through  direct  management  of  the  process. 

Rationale:  The  mandate  of  Red  Deer  College  is  to  primarily  serve  the  region  of  Central  Alberta 
and  Alberta  Education  is  better  positioned  for  the  delivery  of  an  expanded  TLM  to  additional 
high  schools  in  the  province.  The  expertise  that  Alberta  Education  has  in  the  assessment  of 
student  achievement  would  further  assist  in  the  development  of  TLM  and  continue  to  serve 
students,  teachers,  parents,  school  authorities  and  the  province  in  better  meeting  the  individual 
needs  of  all  learners. 
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Appendix  A 

The  Learning  Manager  (TLM)  Online  Standardized  Student  Assessment 
Program  Background 

RDC  has  taken  an  online  course  management  tool  called  The  Learning  Manager  that  is  registered 
by  The  Learning  Management  Corporation  and  Worldwide  Interactive  Network  (which  has  an 
office  in  Calgary)  and  adapted  the  format  for  specific  trades  at  the  Alberta  3400  journeyman 
level  under  the  new  CTS  curriculum  that  is  to  be  implemented  in  two  specific  CTS  curricular 
areas  in  September  2009:  Business,  Administration  Finance  and  Information  Technology  (BIT) 
and  Trades,  Manufacturing  and  Transportation  (TMT).  TLM  has  evolved  over  time  in  various 
software  versions  from  the  original  Computer  Managed  Learning  (CML),  to  the  Learning 
Management  System  (LMS)  and  eventually  the  Curriculum  Information  System  (CIS)  online 
programs,  all  which  have  been  available  for  use  in  colleges,  universities,  schools  and  businesses 
for  in  excess  of  twenty  years. 

TLM  has  been  used  in  other  post-secondary  institutions  in  Canada  to  deliver  distance  learning, 
support  an  online  component  of  traditional  classroom  courses,  deliver  and  secure  online 
assessment,  provide  course  grades  and  build  collaborative  course  materials  and  other  resources 
through  use  of  a  central  repository  of  course  content  information.  An  advantage  that  TLM  has 
over  other  platforms  is  the  assessment  function  that  provides  instant  standardized  assessment  of 
quizzes,  tests  and  exams  and  other  features  which  will  be  explained  in  greater  detail  later  in  this 
document. 

TLM  is  described  as  being  evolved  and  has  evolved  and  directed  to  provide  clients,  either 
corporate  or  educational  organizations  (i.e.  colleges  and  universities)  in  Canada  and  the  U.S., 
with  a  number  of  'advantages': 3 

What  are  the  advantages  of  The  Learning  Manager®  (TLM)  for  our  organization? 
Potentially  significant  cost  savings  through: 

•  Reduced  administrative  and  tracking  costs 

•  Reduced  need  for  learners  to  attend  classes  during  work  hours  thereby  less 
productivity  downtime 

•  Reduced  copying  and  paper  costs  due  to  online  delivery  of  learning  materials 

•  Reduced  facilities  overhead  due  to  less  need  for  classroom  space 

•  Reduced  travel  expenses  due  to  less  need  to  travel  to  training  locations 
Better  resource  and  personnel  utilization  through: 

•  Training  staff  devoting  more  time  to  training  resource  development  rather  than 
delivery 

•  Administrative  staff  spending  more  time  on  other  important  tasks  rather  than 
tracking  training  records  and  performing  other  task  related  to  classroom  training 


3  Taken  directly  from  the  W-WIN.COM  website  (Italics  added),  The  Learning  Manager,  http://wM>w.w- 
win.com/learningmanagementsvstems/ 
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Benefits  of  Web-based  training  include: 

•  Leveraging  existing  technology  resources  to  provide  learners  with  24/7  access  to 
training  materials  so  training  can  occur  anywhere,  anytime 

•  Making  more  learning  resources  available  to  learners  through  online  libraries  - 
anywhere,  anytime 

•  Assuring  the  consistency  of  your  message  because  it's  delivered  the  same  way  to 
all  learners  -  anywhere,  anytime 

•  Delivering  more  training  to  more  learners  with  less  money  and  resource 
commitment  -  anywhere,  anytime 

Other  Benefits  of  the  original  Learning  Management  System®: 

•  Provides  a  full  competency  based  training  solution;  track  skills,  competencies, 
and  non-training-related  data 

•  Understand  your  target  audience  by  performing  online  surveys  of  classroom 
attendees'  learning  levels  prior  to  class  date 

•  Free  video  teleconferencing,  audio  conferencing,  and  presentation  tools  such  as 
desktop  sharing  and  whiteboard  with  the  Live  Action  module 

•  The  Report  Writer  allows  TLM  clients  to  customize  reports  featuring  stored  TLM 
data  on  demand 

•  Anything  that  can  shown  in  an  Internet  browser  can  be  shown  in  The  Learning 
Manager® 

•  A  full  hosting  solution  for  your  Learning  Management  System®  installation  that 
insures  that  your  TLM  application  is  accessible  worldwide,  24  hours  a  day,  7 
days  a  week. 

The  schools  participating  in  this  pilot  have  access  to  TLM  under  a  user  agreement  with  RDC  and 
a  site  fee  that  is  covered  by  Alberta  Education. 

Canadian  Post-Secondary  TLM  Users 
Several  Canadian  post-secondary  institutions  are  currently  using  TLM  to  deliver  locally 
developed  course  material  including: 

•  British  Columbia  Institute  of  Technology  (BCIT) 

•  Canadore  College 

•  Lakeland  College 

•  Laurentian  University 

•  Lethbridge  College 

•  Mohawk  College 

•  Nova  Scotia  Community  College  (NSCC) 

•  Northern  Alberta  Institute  of  Technology  (NAIT) 

•  Red  Deer  College 

•  Red  River  College 

•  Southern  Alberta  Institute  of  Technology 

•  University  of  British  Columbia 

The  number  of  courses  that  are  taught  using  TLM  and  the  number  of  students  accessing  these 
opportunities  varies  from  institution  to  institution.    Some  institutions  have  taken  the  TLM 
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technology  and  adapted  components  to  their  institution  program  and  the  program  has  been  re- 
named in  some  instances.  In  other  instances  some  institutions  offer  online  courses  that  use  a 
combination  of  TLM  and  WebCT4  to  provide  different  applications  for  each  course.  In  many 
programs  it  frequently  appears  that  WebCT  is  used  to  deliver  content  such  as  individual  lessons 
or  units  and  TLM  is  used  in  an  assessment  role  for  quizzes,  tests  and  exams.  TLM  does  possess 
features  that  permit  content  delivery,  though  other  platforms  are  perhaps  better  developed  for 
delivery  and  TLM  focuses  upon  assessment. 

A  sampling  of  the  web  information  on  TLM  use  by  Canadian  post-secondary  institutions  from 
the  above  list  of  institutions  indicates  a  wide  variation  of  online  learning  opportunities  have  been 
integrated  into  their  overall  program  delivery  strategies.  The  institutions  included  in  Table  1 
indicate  that  TLM  is  the  primary  delivery  strategy  for  the  programs  listed.  Many  other  programs 
are  provided  using  other  alternative  delivery  strategies  such  as  WebCT  and  correspondence 
delivery.  It  is  also  recognized  that  a  majority  of  post-secondary  institutions  provide  some  format 
for  online  delivery  learning  opportunities. 


Table  1:  Random  Sample  of  Post-Secondary  Use  of  TLM 


Institution 

Program 

Approximate 
Number  of 
TLM  Courses 

NAIT 

Academic  Upgrading: 
Mathematics  (Pure  Math 
30  equivalent  and 
Technical 

Math/Calculus  courses) 

32 

Liberal  Arts 

1 

Building  Construction 

12 

and  Design 

Health  and  Safety 

1 

Mechanical  and 

15 

Industrial 

Canadore 

Information  Technology 

1 

College 

Technician 

Nova  Scotia 

Science 

4 

Community 
College 

Business  Administration 

11 

Early  Childhood 

.  

11 

Education 

Funeral  and  Allied 

9 

Health  Services 

4  WebCT  was  originally  developed  at  the  University  of  British  Columbia  by  a  faculty  member  in  computer  science 
as  an  online  proprietary  virtual  learning  environment  system  that  is  currently  sold  to  post-secondary  institutions. 
Instructors  can  add  tools  such  as  discussion  boards,  mail  systems,  live  chat  and  content  items  such  as  documents  and 
web  pages.  WebCT  is  now  owned  by  Blackboard,  http://www.blackboard.com/ 
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Prior  Learning 
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Safety  Programs 

3 

Apprenticeship 

222 

TLM  is  used  fairly  extensively  by  post-secondary  institutions,  either  as  a  stand-alone  platform  or 
in  conjunction  with  other  applications.  This  may  be  due  to  post-secondary  institutions 
recognizing  alternate  delivery  methods  as  a  tool  through  which  to  engage  a  variety  of  learners  at 
different  times  and  in  different  locations.  The  financial  resources  and  staffing  requirements  to 
develop  TLM  and  other  online  programs  for  the  locally-developed  courses  appear  to  have  some 
degree  of  priority  with  post-secondary. 

High  School  TLM  User 
There  is  only  one  high  school  in  all  of  North  America  outside  of  this  pilot  project  that  uses  TLM 
to  deliver  course  material  to  students.  Banting  Memorial  High  School,  Simcoe  County  District 
School  Board  in  Alliston,  Ontario  is  a  Grade  9  to  12  school  of  1800  students  in  a  primarily  rural 
area  that  has  utilized  TLM  for  approximately  six  years. 

The  program  at  Banting  Memorial  High  School  began  with  a  teacher  who  was  familiar  with  the 
online  application  and  wished  to  undertake  the  opportunity  for  alternative  delivery  methodology 
in  Math.  In  a  short  period  of  time  all  of  this  teacher's  Math  courses  utilized  TLM  technology. 
Since  the  retirement  of  this  originating  teacher,  TLM  has  been  adopted  by  the  alternative  school 
within  the  school's  regular  program  for  Math  classes  at  grades  9  to  12  that  are  a  combination  of 
courses  that  are  either  locally  developed,  or  at  the  applied,  academic  or  college-preparation  level 
streams  in  the  Ontario  high  school  curriculum.  Currently  there  are  thirty  students  at  this  school 
who  have  access  to  TLM. 

The  school  principal,  when  asked  how  the  students,  parents  and  teachers  have  responded  to 
TLM,  stated,  "They  love  it!"  However,  the  principal  also  provided  a  caution  indicating  that  it 
has  taken  a  considerable  amount  of  time  and  effort  to  create  the  questions  that  are  appropriate  for 
the  grade  level  and  course  and  this  is  the  primary  reason  why  the  program  did  not  originally 
expand  beyond  one  teacher.  The  need  to  review  the  curriculum,  develop  the  appropriate 
question  set  at  the  unit  level  and  validate  them  was  viewed  as,  'Too  much  for  one  school  to 
handle  and  no  other  takers  within  the  district  were  willing  to  assist." 

This  explanation  may  provide  insight  into  why  other  schools  or  districts  have  not  attempted  to 
implement  TLM  at  the  high  school  level. 

Banting  is  considering  expanding  the  TLM  program  on  its  own  once  the  school  becomes  fully 
wireless  and  the  computer  compliment  increases.  The  central  question  in  this  program  vision 
will  be  the  allocation  of  the  staff  and  budget  necessary  to  expand  the  online  offering.  It  is 
interesting  to  note  that  Banting  currently  uses  TLM  as  more  of  a  self-directed  learning  model 
accessed  within  the  school  day  and  has  not  yet  fully  explored  the  opportunity  for  student  access 
at  times  or  locations  outside  of  school  hours. 
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The  Learning  Management  Corporation  and  Worldwide  Interactive  Network  are  not  aware  of 
any  post-secondary  institutions  that  provide  course  material  for  delivery  to  high  school  students 
outside  of  the  pilot  project  that  Red  Deer  College  and  Alberta  Education  are  collaborating  on, 
nor  are  they  aware  of  any  other  high  school  or  K-12  school  authority  in  North  America  using  this 
program. 

Other  online  educational  programs  currently  available  to  Alberta  students  such  as  the  St.  Gabriel 
Cyber  School  through  Greater  St.  Albert  Catholic  Schools  and  Red  Deer  Catholic  Schools  use  a 
different  online  operating  format.  This  program  uses  FirstClass  Client5  and  Skype6  to  provide 
online  educational  delivery  through  email,  text  chat  and  video  conferencing.  All  assessments  are 
completed  through  email  delivery  of  the  quiz,  test  or  exam  from  the  teacher  to  the  student  and 
then  back  to  the  teacher  for  grading.  As  such,  this  system  serves  a  different  purpose  than  TLM 
and  does  not  have  standardized  assessment  capability. 


Commercial  service  provided  by  Open  Text  Corporation  that  offers  student  email,  social  networking,  collaborative 
conferences  and  work  spaces,  assignment  calendars  and  scheduling,  podcasting  and  file  storage  for  educational 
users. 

http://www.firstclass.com/ResourcesFolder/ResourcesPDFs/FirstClass%20Collaboration%20Suite%20Education.pd 
f/?Plugin=FC 

6  Commercial  service  provided  by  Skype  Limited  that  offers  audio  contact,  instant  messaging,  video  call  and 
conferencing  capabilities,  http://www.skype.com/business/features/calling/ 
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Appendix  B 


Carpentry  and  welding  Alignment  Charts:  the  new  3400  level  (introduction  September  2009) 
to  the  current  CTS  courses 

The  current  POS  are  on  the  vertical  axis  and  the  new  3400  courses  are  on  the  horizontal  axis. 
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Appendix  C 

TLM  Welding  Screen  Shot 

This  screen  shot  indicates  some  of  the  modules  that  are  available  for  welding.  The  green  band 
indicates  modules  that  students  are  able  to  access  initially,  allowing  them  to  then  proceed  down 
the  vertical  axis,  writing  a  quiz  for  each  module.  The  red  band  for  3404  indicates  a  course  that 
the  student  must  receive  permission  from  the  CTS  teacher  in  order  to  begin.  If  a  student  fails  a 
review  quiz  three  times  for  any  module,  they  are  'locked  out'  until  they  meet  with  their  CTs 
teacher. 
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Appendix  D 

Pilot  Teacher  and  Student  Interview  Responses 

Individual  interviews  were  conducted  with  each  teacher  involved  in  the  pilot,  a  number  of 
participating  students  and  with  school  administrators.  One  administrator  provided  email 
responses  to  the  questions  that  have  been  incorporated  into  the  overall  responses.  As  such,  the 
set  questions  for  all  teacher  and  administrator  respondents  are  presented  as  follows: 

1 .  How  has  the  online  TLM  program  benefitted: 

(a)  You  as  a  teacher 

(b)  Your  students 

2.  What  value  do  you  see  for  use  of  this  online  program  by  non-journeyman  CTS  teachers? 

3.  What  do  you  believe  is  the  potential  for  expansion  of  this  online  resource? 

Teacher  Responses 

1.  How  has  the  online  TLM  program  benefitted: 

(a)  You  as  a  teacher 

the  online  system  works  well 

the  school  authority  is  willing  to  upgrade  the  facility  to  make  it  more  usable 

-  access  to  strong,  pre-made  resources  that  are  current 

-  an  almost  one-stop  resource 
the  software  is  excellent 
supervised  test  is  a  'snap' 

the  program  can  be  confusing  when  not  used  all  the  time 

-  may  increase  the  workload  for  teachers 

-  this  program  could  replace  the  regular  program 

-  good  relationship  with  RDC 

see  3400  courses  as  an  injection  of  energy  to  continue  to  offer  students  relative  learning 
opportunities  at  outreach  and  TLM  has  been  a  great  assist 

number  one  is  assessment  standards... the  link  to  advanced  learning  makes  the  testing 
respectful  and  meaningful 

-  the  ease  of  operation  is  appreciated 

(b)  Your  students 

students  might  be  floundering  because  they  are  on  their  own  to  work  on  the  modules 
o   the  lay-out  of  the  lab  is  the  main  problem  due  to  the  lack  of  classroom  space  and 

computers.  Students  have  to  use  the  library  and  the  distance  and  the  lack  of  me 

being  there  to  work  with  them  is  a  significant  issue 
students  are  very  interested  in  being  able  to  move  ahead 
some  students  are  very  motivated  to  take  the  RDC  program 

-  they  have  access  to  up-to-date  resources  for  their  trade 

-  all  materials  are  to  industry  standards 

-  helps  to  challenge  the  students  who  are  capable 

some  of  the  wording  in  the  questions  is  not  clear  and  in  a  few  cases  there  was  not  any 
connection  between  the  test  questions  and  the  written  module  work 
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not  enough  time  for  a  balance  of  the  theoretical  and  practical  work 
students  need  to  have  a  regular  study  period  scheduled  to  assist  them 
time  is  a  factor  when  trying  to  complete  the  theoretical  and  practical  work 
some  students  see  it  as  balanced 
need  to  schedule  separate  lab  and  classroom  time 

problems  with  the  online  program  at  the  start  of  the  semester,  but  they  seem  to  have  been 
taken  care  of.  It  took  a  few  weeks  to  get  them  ironed  out 
the  TLM  has  been  a  strong  tool  for  our  outreach  student 

aspects  of  the  online  test  have  been  a  'wake-up'  call  for  our  student  and  he  has  responded 
well 

our  outreach  student  is  proud  of  his  work  with  the  online 

2.  What  value  do  you  see  for  use  of  this  online  program  by  non-journeymen 
CTS  teachers? 

it  really  depends  upon  AIT  and  how  they  approach  non-journeymen 
you  really  do  not  need  journeymen  to  teach  1st  period  apprenticeship 
non-journeymen  are  provided  additional  resources  that  could  be  beneficial 
non-journeymen  teachers  tend  to  set-up  courses  so  that  their  students  avoid  any  problems, 
whereas  journeymen  are  taught  to  teach  problems  to  allow  students  to  develop  skills  to 
work  through  them 

students  need  to  see  and  understand  the  dangers  involved  in  their  trade 
gives  them  exposure  to  industry  material 
access  to  industry  test  bank  questions 

you  cannot  buy  this  resource  like  you  can  with  other  subject  areas 
these  are  stronger  questions  to  challenge  students 
very  time  consuming  to  construct  your  own  tests 
RDC  has  done  a  great  job 

the  average  non-journeymen  may  need  additional  assistance 
non-journeymen  teachers  learn  on  the  job  to  try  and  stay  ahead  of  their  students 
concessions  need  to  be  made  to  encourage  journeymen  into  the  education  program 
ensure  that  all  supporting  information  is  provided  for  the  modules 

concern  with  the  gap  in  students  coming  from  other  schools  (their  previous  90%  is  the 
equivalent  of  a  45%  in  my  program... they  just  do  not  have  the  skills  that  they  are 
supposed  to  if  they  have  already  finished  the  modules  that  their  previous  school  said  that 
they  did) 

TLM  is  beneficial  to  anyone 

non-journeymen  need  additional  resources  and  support 
o    PD  in  the  summer  is  a  must 
o    Who  pays  for  the  support 
o   non-journeymen  need  to  take  courses 

3.  What  do  you  believe  is  the  potential  for  expansion  of  this  online  resource? 

great  potential  to  expand  under  the  right  lab  and  facility  conditions 
can  be  a  challenge  for  non-journeymen 
may  push  these  non-journeymen  to  expand  their  own  skills 
strong  practical  PD  for  these  teachers  would  be  required 
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lack  of  journeymen  to  ask  for  assistance  is  an  issue  for  non-journeymen 

-  the  mark  gap  between  some  schools  is  overwhelming. .  .where  do  they  get-off  giving  such 
high  marks  to  a  student  who  cannot  complete  the  skill? 

o    some  marks  are  just  not  justified  and  this  is  causing  us  some  significant  issues 
when  students  from  other  schools  transfer  into  our  program 

-  the  potential  to  expand  this  is  significant 

-  issues  at  the  3400  level  with  the  few  number  of  students  who  are  able  to  take  this 

-  need  to  expand  the  resources  to  make  this  work  in  a  small  school 

-  need  to  expand  the  resources  that  are  available  on  TLM  that  can  be  downloaded  for  class 
(i.e.  videos,  pictures,  assignments) 

-  funding  is  an  issue  especially  in  small  school 

o    central  office  is  skimming  a  lot  of  money  off  of  the  top  and  it  is  making  it  tough  at 
the  schools 

o    there  are  far  too  many  people  working  in  central  office  in  a  board  that  is  so  small 

-  limited  by  the  facility  and  instructors 
this  program  could  expand  to  other  trades 

-  needs  to  be  monitored  closely 

could  be  a  plus  to  keep  more  students  in  school 

-  Gr.  1 0s  are  excited  to  hear  about  this  for  their  Gr.  1 1  year 

This  could  be  and  should  be  expanded  to  all  other  course  levels  in  CTS 

-  The  3400  level  welding  course  through  TLM  is  a  very  challenging  course  with  high 
expectations  of  the  learner.  Students  quickly  figure  out  that  there  is  an  abundance  of 
material  that  needs  to  be  learned  and  understood.  It  is  an  eye-opener  for  students  who 
have  no  idea  what  to  expect  from  a  college  level  course. 

From  what  I  have  witnessed  students  will  rise  to  the  challenge  knowing  that  more  is 
expected  of  them.  The  difficulty  level  gives  credibility  to  the  vocation  and  makes 
students  realize  just  how  much  an  apprentice  welder  is  expected  to  know. 

Student  Responses 

Conversations  with  students  were  more  informal  and  did  not  follow  a  set  of  pre-determined 
questions,  but  were  led  by  the  student  responses  to  previous  questions.  The  responses  for  one 
student  were  forwarded  electronically  by  the  school  administrator  and  have  been  incorporated 
into  the  other  interview  responses. 

Not  all  student  conversations  involved  those  who  were  participating  in  the  pilot  as  some  students 
were  aware  of  TLM,  but  were  not  part  of  the  program.  Some  conversations  with  specific 
students  took  place  due  to  their  previous  experiences  at  other  high  schools  or  to  gauge  their 
perceptions  of  a  program  that  was  potentially  "down  the  road"  for  them. 

Student  responses: 

-  has  taken  all  of  the  welding  courses  at  school  that  he  possibly  can 

-  the  RDC  modules  are  challenging  and  he  enjoys  the  opportunity 

-  a  lot  of  reading,  but  he  knows  that  it  is  part  of  it 

-  has  been  taking  welding  at  school  since  Grade  9,  but  he  has  been  welding  at  home  all  of 
his  life 

-  has  learned  all  of  the  industry  safety  guidelines  that  he  did  not  know  before 
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some  industry  standards  in  other  areas  are  new  to  him 
it  has  been  a  lot  of  work 

I  am  going  to  apprentice  at  a  shop  in  Calgary  and  have  already  been  offered  a  job  with 
them 

need  to  have  computers  in  the  shop  because  it  is  not  good  when  I  have  a  question  and  the 
shop  is  on  the  other  side  of  the  school 
challenging  and  a  lot  of  reading 

it  is  tough  to  get  the  work  done  in  class  and  to  then  get  all  of  the  reading  done  at  home 
likes  the  challenge 

it  is  neat  that  I  am  able  to  do  this  while  still  in  high  school 

my  friends  want  to  take  this  next  year 

a  standardized  test  means  that  this  is  an  important  subject 

the  questions  are  tough,  but  I  can  then  read  through  the  book  again  to  find  it  and  it  is  a  lot 

easier  to  do  this  than  some  of  the  other  classes 

there  just  isn't  enough  time  in  class  to  get  it  all  done 

it  is  laid-out  and  easy  to  follow  for  the  most  part 

my  teacher  is  great  with  it 

like  the  modules 

the  modules  are  easy  to  understand 

I  like  it  immediately  telling  me  what  I  got  right  or  wrong  and  going  online 
the  tests  are  worded  so  I  can  understand  the  questions 

this  is  not  easy  stuff.  You  have  to  study  and  know  it  or  you  will  not  pass.  The  online 
tests  are  tough  and  it  is  an  exam  that  means  a  lot 

I  don't  know  how  I  got  the  (high)  marks  that  I  did  at  my  last  school.  I  really  didn't  do 
anything  in  class 

I  am  looking  forward  to  being  able  to  take  that  next  year.  They  are  doing  what  I  want  to 
do 

it  sounds  like  a  tough  class,  but  they  are  doing  some  neat  things 
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Appendix  E 


Draft  CTS  Student  Assessment  Framework 

The  potential  measures  to  indicate  the  impact  of  the  delivery  and  implementation  of  the  Revised 
CTS  Program  are  in  draft  form  in  the  following  three  tables  and  are  taken  from  the  document 
Career  and  Technical  Studies  (CTS)  Student  Assessment  Framework:  Focus  Upon  Two  Tasks, 
Draft  Document  12.1  posted  on  the  CTS  Education  Base  website  internal  to  Alberta  Education. 


Outcome 

Measure 

A  more  focused  program  of 
studies 

•  %  of  high  school  graduates  earning  credits  in  CTS  (AE 
data) 

•  Average  credits  earned  by  high  school  students,  by  type 
of  course  work  in  CTS  (AE  data) 

•  %  of  student  satisfaction  with  the  CTS  program  taken 
(Survey) 

•  %  of  students  satisfied  with  the  CTS  strands  offered  at 
their  school  (Survey) 

Updated  resources 

•    %  of  CTS  teachers  satisfied  that  the  curricular  resources 
provided  to  them  meet  the  needs  of  their  students  in  the 
new  program  (Survey) 

Enhanced  access  to 
programming 

•  Number  of  students  who  have  a  course  completion 
record  in  CTS  (AE  data) 

•  Average  credits  earned  by  high  school  students  in  CTS, 
by  grade  (AE  data) 

•  Average  credits  earned  by  high  school  students,  by 
subject  area  (AE  data) 

•  %  of  high  school  graduates  who  enrolled  in  post- 
secondary  education  or  training  within  2  years  after 
graduation  (AE  data) 

•  Number  of  external  credentials  that  are  available  for 
students  (School  and  jurisdiction  data) 

•  Number  of  external  partnerships  developed  between 
districts  and  other  organizations  (School  and  jurisdiction 
data) 

•  Number  of  schools  registered  as  a  Skills  School  through 
Skills  Canada  (Skills  Canada  data) 

Professional  learning 
opportunities  for  CTS 
teachers 

•  %  of  CTS  teachers  reporting  use  of  computer  and 
technology  related  instruction  to  enhance  their 
professional  learning  (Survey) 

•  %  of  CTS  teachers  reporting  that  their  professional 
learning  has  improved  the  learning  opportunities  of  their 
students  in  the  new  CTS  program 

•  %  of  teacher  satisfaction  with  pre-service  preparation 
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for  CTS  (Survey) 

•  %  of  teacher  satisfaction  with  in-service  preparation  for 
CTS  (Survey) 

•  %  of  teachers  trained  using  a  variety  of  assessments 
including  use  of  rubrics  to  measure  performance 
(Survey) 

Certification  of  specialist 
teachers 

•  Number  of  teachers  teaching  CTS  courses  (School  data) 

•  %  of  time  that  teachers  spend  teaching  CTS  (FTE) 
(School  data) 

•  Number  of  teachers  trained  in  (university  major  or 
minor)and  teaching  in  CTS  in  the  province  (School  data) 

•  Number  of  CTS  teachers  who  have  journeyman 
certification  (School  data) 

•  %  of  teachers  holding  specialized  credentials  in  areas  of 
CTS  program  delivery  other  than  journeyman  (School 
data) 

•  Number  of  teachers  supervising  CTS  instructors  (School 
data) 

•  %  of  FTE  involved  for  teachers  supervising  CTS 
instructors  (School  data) 

•  Qualifications  of  the  teacher  supervising  CTS  instructors 
(School  data) 

Facilities  and  equipment  to 
acceptable  standard 

•  %  of  CTS  labs  that  have  experienced  structural  upgrades 
(School  data) 

•  %  of  equipment  ever-greened  in  the  school  CTS  labs 
(Survey) 

•  Number  of  shared  facility  arrangements  for  CTS  labs 
between  schools  or  other  organizations  (School  data) 

•  Number  of  mobile  CTS  labs  in  use  by  jurisdictions 
(Jurisdiction  data) 

The  potential  measures  for  the  assessment  of  student  achievement  in  the  Revised  CTS  Program 
are  in  draft  form  and  presented  as  follows: 

Outcome 

Measure 

•  Student  self-assessment  of  their  learning,  administered  at 
the  completion  of  each  module  (Direct  measure, 
developed  by  classroom  teacher) 

•  Assessment  of  a  student  learning  portfolio  submitted  at 
the  end  of  a  semester  program  (i.e.  4-5  modules) 
(Indirect  measure,  developed  by  classroom  teacher, 
school  and/or  jurisdiction) 

•  %  of  students  satisfied  with  assessment  practices  used  in 
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CTS  courses  (Survey) 

•     ®/n  nf  teaphers  nsinc*  nprformanrp  data  (\  p  needs 

assessment,  performance  measures,  student  surveys)  to 

improve  student  achievement  (Survey) 

•     °/n  of  students  romnletinp  assignments  that  renmre  them 

w              u   VJX   J  L  L4  vj.  Vv  1  llj  vVlllU  1  w  t  111       Ul3l3 1  E^lll 11  Vvll  to   null    I                   V  lllvlll 

to  demonstrate  relevant  academic  skills  and  to  solve 

problems  through  projects  (Survey) 

•    %  of  students  able  to  incorporate  academic  and  technical 

skills  needed  within  the  occupational  area  of  study 

(Survey) 

•    %  of  students  completing  apprenticeship  courses  in  CTS 

(AE  data) 

•    Number  of  students  that  complete  a  credentialed 

pathway  (AE  data) 

•    Number  of  students  that  complete  an  external  credential 

(School  data) 

•    High  School  completion  rates  calculated  on  a  3,  4  and  5 

year  cycle  (AE  data)7 

•     Student  absenteeism  rate  f  School  dataV* 

•    Number  of  students  who  attained  the  level  of  excellence 

in  regional,  provincial  and  national  skills  competitions 

(Skills  Canada  data) 

•     XVip  miml"ipr  r>f* QtiiHents  who  aeeess  a  online  test 

w         1 11C  llUlllUvvl  VJ1  OlUUdllo  WllVJ                     d  vjllllllvv  Ivvol 

provided  by  the  Regional  Trades  Initiative  in  carpentry 

or  welding  following  the  completion  of  a  grouping  of 

courses  (Red  Deer  College  data) 

7  And 9:  These  measures  are  not  intended  to  indicate  that  any  improvement  in  school  completion  or  student 
absenteeism  rates  are  the  sole  result  of  a  CTS  initiative  taken  in  isolation.  Improvements  in  the  rate  of  these  two 
measures  would  be  a  reflection  of  multiple  strategies  developed  by  many  education  and  community  stakeholders,  of 
which  CTS  does  have  an  opportunity  to  impact  in  a  positive  manner.  It  is  also  noted  that  many  student  CTS 
assessment  measures  are  a  reflection  of  learning  opportunities  obtained  over  the  course  of  a  young  person's 
educational  career  and  life  experiences. 
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